Fruit and vegetables (FV) are the major source of bioactive compounds for human beings. FV supply chains are complex and sensitive due to various features, including the seasonality of products, variations in demand, and short shelf-lives. The amount of waste in FV supply chains is significant compared with other supply chains as 44% of fresh FV produced globally are wasted in the food chain. This large amount of waste has a significant impact on the economy, food security, available natural resources, and the environment. To reduce food losses in the fresh food supply chain (FFSC), the root causes of waste must be first identified. While a number of researchers have investigated food losses in Australia, most only consider a specific stage in the supply chain and multiple stages in the FFSC are often overlooked. Additionally, the impact of advanced storage technologies, packing, handling, and transport on food losses should be investigated. Furthermore, supply chain practices are changing in response to uncertainties, such as the pandemic and climate changes, which also need to be captured. This research aims to identify the key factors contributing to fresh fruit and vegetable losses through a comprehensive empirical study. Primary data were collected through a well-designed questionnaire-based survey targeting major stakeholders in the FFSC, including farmers, distributors, and retailers in Australia. The survey investigates current postharvest practices and the effects of these practices on food losses. The main factors influencing food losses were identified and the options to reduce these losses were outlined. The results showed that losses mostly occurred at the farm level, and picking practices and preharvest conditions largely contributed to FV losses. The results highlight the need for proper training and education for workers involved in harvesting and handling fresh produce.