You wouldn’t need anything that extravagant, you could reasonably find all these people in California in the late 1800’s. The earliest Japanese immigrants to California happened in the 1860’s. After the gold rush people from all over the world flocked to Cali.
It’s actually pretty plausible, the first wave of Japanese people to immigrate to California and Hawaii was in the 1860s. By the 1900 census there were nearly 25k Japanese people living on the West Coast.