Off timing of a building in the "Dead Man's Chest" movie

1. The weathering of the house is perfectly consistent with the timeframe provided by e.g. Port Royal's (where Will and Elizabeth are from) minimum age. In no way would people be able to build this town up from nothing in less time than it takes for that building to turn dilapidated.

2. Keep in mind the effect that salt water has on wood, it dramatically speeds up the damage and weathering that timber endures when it is not maintained properly. While the building was not by the waterside, it's on an island in a humid climate. There's salt and humidity in the air and it's going to crack the timber over time, leading to more exposed internal timber and accelerated rotting/weathering.

3. Pirates of the Caribbean is not a historical documentary and does not attempt to be precise about the historical context of the Caribbean. There are plenty of anachronisms in the movies.

4. Even then, the movies take place around 1740. Colonies started being built from 1493 on (one year after Columbus' voyage), and in the early 1600s the Spanish started spreading colonies across the Caribbean (second paragrah). That's more than enough time for this building to be constructed, lived in, abandoned, and end up in the state that it's in during the movie in the so-called 1740's.

Though I fully agree with Flater's explanation, there's an addidional aspect in 'historical' movie-making, which is "if it's old it has to look old".

This is especially noticeable in things from the Tudor period [The Tudors, Wolf Hall etc], as many of the key buildings still survive & are often used in filming.

The buildings are, of course, now 500 years old & whilst in remarkable condition for their age - due to public interest & spending to keep them that way - they do actually look 500 years old.

When anyone builds interior sets to match, they do tend to 'brown everything down a bit' to keep that look.
Using our earlier Tudors/Wolf Hall comparison, The Tudors very much does this 'browning down' on the interiors - a kind of over-cooked sepia look which makes it all look very 'period'. Wolf Hall, on the other hand, is shot almost entirely on location, often with a lot of natural light, & so doesn't have this extra-dark sepia look to it.
In both cases, though, the buildings still look 'old' even though at the time the events took place, they'd have been almost brand-new.

I can't post pictures, for obvious legal reasons, but wait for The Great [Hulu, mid-May] & check out the interiors on that. The theme in a lot of the sets is really just 'brown on brown' with banks of massive 12k lights streaming through the 'outdoor' windows.