At the same time, a sim, any sim, is not the appropriate tool to compare things like "how much damage does weapon X on this job do compared to weapon Y on this other job in this content", simply due to the number of variables the sim has to fix in place to get results that make sense, and how much the sim simply ignores because they are hard to model.
This is true, but nobody uses sims in this way afaik. We don't have the exact stats of Sortie, Omen, or Odyssey bosses, so it can't determine that.
What it can do is calculate average damage over a common enemy that we do know the stats for, with a set of variables (buffs, job traits and abilities, and gear sets) specific to the test. Then, by changing out only the weapon (or a few items in a set), you can see differences in the average damage output of each by tweaking the tool slightly (adding this buff, absorb stat, etc, removing this piece in favor of another). This gives you a very close estimation of comparing singular items like weapons against each other because the environment is extremely controlled and the variables you are changing are only a few, so it is easy to see where the differences occur. This kind of control is actually preferred, because if you go out into the field and whack a mob 100 times, even with near-perfect tracking, you can still be fooled by the eyeball test, or a mistake can happen, a buff can fall off you get enfeebled, or some other factor, which can throw the entire test off. It would also take entirely too long to actually know for sure if something was a better option than another.
The sim is a tool for comparing one set to another set and seeing if/how much of an improvement it makes, not jobs or weapons.
I can understand what you mean that a sim is not specifically a job-comparison tool (but it can even be used for that since the tool doesn't know the difference between a job and gear, its all inputs/numbers), but I disagree with the weapon. A weapon is just a piece of equipment, no different than a body armor. You can simulate damage output on weapons just fine and compare them to one another, using the controlled approach I mentioned above. Again, this is way more reliable than the eyeball test.
The last thing anyone should be doing in a thread about opinion tier lists for weapons, a troll list from its inception, is bringing maths into it. Opinion, obviously.
Really? I get that the entire premise of a tier list is a person's individual opinion, but there is also something considered an "agreed upon" fact from peers as to where items are placed. An example is Naegling, it's so obvious that nobody could
reasonably have the opinion that it is not worthy of its placement (in an "opinion" ranking nonetheless). So if 100% of the people have the Opinion that Naegling is S tier, it becomes basically a fact among everyone. Where it gets dicey is when people
don't agree on placements at the extreme tiers (A, D). Nobody really fusses over tier placements in the middle rankings, because those can flip flop based on the wind or the time, but the extreme "this is OP"/"this sucks" is more disputed than anywhere else, because people have individual experiences and uses cases they can bring to the discussion that they feel heavily weigh into the placement. From that point, the tier lists just become a huge conversation about people's input about the weapons, which is fine. But that doesn't means someone can't contribute evidence, whether its from a sim or a personal use-case, to help others understand where they might be mistaken about a particular weapon placement. Being ignorant about a weapon is common; not everyone is going to have every single weapon on every single job, so it is perfectly reasonable for someone to share their perspective to educate others who are misinformed.
Now where I will agree is that there are a number of factors involved that comes into play when it comes to using Sims. I believe humans should be smart enough to take the information from the sim and deduce what is the best choice to make based on the information provided
on their own; you wouldn't let the tool tell you exactly how you should gear for every particular scenario. That would be just as silly as you asking an AI a question and assume it is 100% correct, and blindly taking it's word on the matter.