Americans throw away almost as much food as they eat because of a “cult of perfection”, deepening hunger and poverty, and inflicting a heavy toll on the environment.

Vast quantities of fresh produce grown in the US are left in the field to rot, fed to livestock or hauled directly from the field to landfill, because of unrealistic and unyielding cosmetic standards, according to official data and interviews with dozens of farmers, packers, truckers, researchers, campaigners and government officials.

From the fields and orchards of California to the population centres of the east coast, farmers and others on the food distribution chain say high-value and nutritious food is being sacrificed to retailers’ demand for unattainable perfection.

“It’s all about blemish-free produce,” says Jay Johnson, who ships fresh fruit and vegetables from North Carolina and central Florida. “What happens in our business today is that it is either perfect, or it gets rejected. It is perfect to them, or they turn it down. And then you are stuck.”

Food waste is often described as a “farm-to-fork” problem. Produce is lost in fields, warehouses, packaging, distribution, supermarkets, restaurants and fridges.

By one government tally, about 60m tonnes of produce worth about $160bn (£119bn), is wasted by retailers and consumers every year – one third of all foodstuffs.

But that is just a “downstream” measure. In more than two dozen interviews, farmers, packers, wholesalers, truckers, food academics and campaigners described the waste that occurs “upstream”: scarred vegetables regularly abandoned in the field to save the expense and labour involved in harvest. Or left to rot in a warehouse because of minor blemishes that do not necessarily affect freshness or quality.

When added to the retail waste, it takes the amount of food lost close to half of all produce grown, experts say.

“I would say at times there is 25% of the crop that is just thrown away or fed to cattle,” said Wayde Kirschenman, whose family has been growing potatoes and other vegetables near Bakersfield, California, since the 1930s. “Sometimes it can be worse.”

“Sunburnt” or darker-hued cauliflower was ploughed over in the field. Table grapes that did not conform to a wedge shape were dumped. Entire crates of pre-cut orange wedges were directed to landfill. In June, Kirschenman wound up feeding a significant share of his watermelon crop to cows.

Researchers acknowledge there is as yet no clear accounting of food loss in the US, although thinktanks such as the World Resources Institute are working towards a more accurate reckoning.

Imperfect Produce, a subscription delivery service for “ugly” food in the San Francisco Bay area, estimates that about one-fifth of all fruit and vegetables are consigned to the dump because they do not conform to the industry standard of perfection.

But farmers, including Kirschenman, put the rejection rate far higher, depending on cosmetic slights to the produce because of growing conditions and weather.

 That lost food is seen increasingly as a drag on household incomes – about $1,600 a year for a family of four – and a direct challenge to global efforts to fight hunger, poverty and climate change.

Globally, about one-third of food is wasted: 1.6bn tonnes of produce a year, with a value of about $1tn. If this wasted food were stacked in 20-cubic metre skips, it would fill 80m of them, enough to reach all the way to the moon, and encircle it once.

The Obama administration and the UN have pledged to halve avoidable food waste by 2030. Food producers, retail chains and campaign groups such as the Natural Resources Defense Council have also vowed to reduce food loss in the ReFED initiative.

Food experts say there is growing awareness that governments cannot effectively fight hunger, or climate change, without reducing food waste. Food waste accounts for about 8% of global climate pollution, more than India or Russia.

“There are a lot of people who are hungry and malnourished, including in the US. My guess is probably 5-10% of the population are still hungry – they still do not have enough to eat,” said Shenggen Fan, the director general of the International Food Policy Research Institute in Washington. “That is why food waste, food loss matters a great deal. People are still hungry.”

That is not counting the waste of water, land and other resources, or the toll on the climate of producing food that ends up in landfill.

Within the US, discarded food is the biggest single component of landfill and incinerators, according to the Environmental Protection Agency. Food dumps are a rising source of methane, a far more powerful greenhouse gas than carbon dioxide. But experts readily acknowledge that they are only beginning to come to grips with the scale of the problem.

The May harvest season in Florida found Johnson with 11,000kg (24,250lbs) of freshly harvested spaghetti squash in his cool box – perfect except for brown scoring on the rind from high winds during a spring storm.

“I’ve been offering it for six cents a pound for a week and nobody has pulled the trigger,” he said. And he was “expecting an additional 250,000lbs of squash,” similarly marked, in his warehouse a fortnight later.

“There is a lot of hunger and starvation in the United States, so how come I haven’t been able to find a home for this six-cents-a-pound food yet?” Johnson asked.

Such frustrations occur regularly along the entirety of the US food production chain – and producers and distributors maintain that the standards are always shifting. Bountiful harvests bring more exacting standards of perfection. Times of shortage may prove more forgiving.

Retail giants argue that they are operating in consumers’ best interests, according to food experts. “A lot of the waste is happening further up the food chain and often on behalf of consumers, based on the perception of what those consumers want,” said Roni Neff, the director of the food system environmental sustainability and public health programme at the Johns Hopkins Center for a Livable Future in Baltimore.

 “Fruit and vegetables are often culled out because they think nobody would buy them,” she said.

 But Roger Gordon, who founded the Food Cowboy startup to rescue and re-route rejected produce, believes that the waste is built into the economics of food production. Fresh produce accounts for 15% of supermarket profits, he argued.

 “If you and I reduced fresh produce waste by 50% like [the US agriculture secretary] Vilsack wants us to do, then supermarkets would go from [a] 1.5% profit margin to 0.7%,” he said. “And if we were to lose 50% of consumer waste, then we would lose about $250bn in economic activity that would go away.”

Some supermarket chains and industry groups in the US are pioneering ugly produce sections and actively campaigning to reduce such losses. But a number of producers and distributors claimed that some retailing giants were still using their power to reject produce on the basis of some ideal of perfection, and sometimes because of market conditions.

The farmers and truckers interviewed said they had seen their produce rejected on flimsy grounds, but decided against challenging the ruling with the US department of agriculture’s dispute mechanism for fear of being boycotted by powerful supermarket giants. They also asked that their names not be used.

“I can tell you for a fact that I have delivered products to supermarkets that was [sic] absolutely gorgeous and because their sales were slow, the last two days they didn’t take my product and they sent it back to me,” said the owner of a mid-size east coast trucking company.

 “They will dig through 50 cases to find one bad head of lettuce and say: ‘I am not taking your lettuce when that lettuce would pass a USDA inspection.’ But as the farmer told you, there is nothing you can do, because if you use the Paca [Perishable Agricultural Commodities Act of 1930] on them, they are never going to buy from you again. Are you going to jeopardise $5m in sales over an $8,000 load?”

 He said he experienced such rejections, known in the industry as kickbacks, “a couple of times a month,” which he considered on the low side for the industry. But he said he was usually able to sell the produce to another buyer.

 The power of the retail chains creates fear along the supply chain, from the family farmer to the major producer.

 “These big growers do not want to piss off retailers. They don’t enforce Paca on Safeway, Walmart or Costco,” said Ron Clark, who spent more than 20 years working with farmers and food banks before co-founding Imperfect Produce.

 “They are just not going to call because that will be the last order they will ever sell to them. That’s their fear. They are really in a pickle.”