Where the aircraft is registered is only the concern of the country hosting the aeronautical activity, not the FAA for a part 61 perspective.
For the most part, you can do all of your training outside the U.S. You run into issues taking knowledge tests and finding DPEs (they’re out there, authorized outside the U.S., but far fewer than in the US). For instrument work, part 61 regulations currently define an instrument approach as a procedure defined in part 97. Instrument approaches outside the U.S. do not fall under part 97. I suspect this is often inadvertently violated and I think the FAA may even be working to address this, but I’m not certain.