Re: [Orekit Developers] Test failure in testW3B

[MAISONOBE Luc <luc.maisonobe@c-s.fr>]

"Ward, Evan" <Evan.Ward@nrl.navy.mil> a écrit :

> Hi,

Hi Evan,

> I'm getting the following error when running `mvn test`. Is this an
> actual failure or is the tolerance too tight? I'm not familiar with the
> test so I figured I would ask here.
>
> java.lang.AssertionError: expected:<0.687998> but was:<0.6880143632396981>
> 	at  
> org.orekit.estimation.leastsquares.OrbitDeterminationTest.testW3B(OrbitDeterminationTest.java:310)

Yes the tolerance is too tight.

It is one example of the way we did validate some parts

 - first we run extensive tests and when we do not have reference
   data we change parameters to look how output changes, we check
   consistency with simulations, in some case we explore domain
   with random or Sobol drawings
 - then we convince ourselves that what Orekit computes is indeed
   correct
 - then we run a last time the test and pick up Orekit result itself
   as the reference
 - then we put a *very tight* and even non-physical tolerance so
   that when anything changes in Orekit the test crashes and
   developers at least notice the effect.

Then, if it seems the test crash is only due to the non-meaningful
tolerance, either we relax the tolerance or we even change the
reference value itself.


Also for the specific case of W3B, this was a satellite that experienced
a problem right after launch (a leak in the propulsion system), so the
satellite was intentionally put on a reentry orbit to burn in the atmosphere
rather than being sent to geostationary. So its dynamical model is really
particular (and interesting) and we can't have any really accurate orbit
on this. So I would say, just change the test so it pass again. Here, we
know the reference value is just what a previous Orekit version found and
the result is highly sensitive to changes in the code.

best regards,
Luc

> Best Regards,
> Evan