Amid simmering tension between China and self-ruled Taiwan,
Beijing suspended purchases of grouper fish, Chen's main product, in June after repeatedly detecting banned chemicals.
But the geopolitical storm that followed Pelosi's visit on Aug. 2 is likely to put the livelihoods of Chen and other farmers and fishermen in the island's southern Pingtung county at further risk.
"Our job is to do fish farming," Chen told Reuters. "We only hope that once the fish grow bigger, we will sell them."
"Because of political issues and our government not having a communication channel, the Chinese government is blocking our economy," he said.
Beijing extended its ban on Taiwan's seafood a day after Pelosi's arrival, blocking shipments of chilled white striped hairtail and frozen horse mackerel, saying it had found traces of the coronavirus on the packaging of some products.
The bans were widely seen as retaliation against Taiwan for allowing the visit, which prompted China to launch unprecedented military exercises around the island. China described Pelosi's visit as a provocation that undermines its sovereignty and territorial integrity.
Taiwan rejects China's sovereignty claims and says only its 23.5 million people can decide their future.
"I personally don't think it helped Taiwan whatsoever," Chen said of Pelosi's visit. "It creates some economic loss for the Taiwanese people instead. I don't know why she came."
China also banned imports of sand - used to make concrete - and suspended shipments from 35 Taiwanese exporters of biscuits and pastries.
In the first half of this year, China's imports from Taiwan reached a value of $122.5 billion, up 7.3% from a year earlier, Chinese customs data showed. Taiwan's sales of fish and other aquatic invertebrates to China amounted to 399 million yuan ($59 million).
According to Goldman Sachs, food exports to China accounted for just 0.4% of China-bound Taiwan exports, which are dominated by high-tech products, and sanctions have mostly affected farmers and fishermen.
Taiwan's fishermen have been looking for other options. Zheng Rui-Long, the owner of a fish processing factory in Pingtung, told Reuters he was trying to supply fish for bento boxes for Taiwan's railway network.
"Fishermen here indeed are worried that they don't know where to sell groupers," he said.
"People need to live. We don't understand too much about politics but we only need a kind and peaceful relationship between China and Taiwan and also to co-exist."
(Writing by David Stanway; Editing by Robert Birsel)
By Ann Wang and I-Hwa Cheng