Re: [Orekit Developers] Test failure in testW3B

["Ward, Evan" <Evan.Ward@nrl.navy.mil>]

Hi Maxime,

On Thu, 2018-03-08 at 12:22 +0100, JOURNOT Maxime wrote:

> Hi Evan, Luc,
>
> I was about to answer but you were faster Luc.
> Just one weird thing on my side: I cannot reproduce the bug on my machine with the latest version of the "develop" branch.
> Is that a numerical discrepancy (it seems quite big though) or should we investigate further ?

It happens every time for me. I also have a Jenkins instance that builds the latest Orekit code every night and testW3B has been failing every build since August.

Perhaps a localization issue? Though I don't know how that would affect it.

Best Regards,

Evan

Best regards,
Maxime

Le 07/03/2018 à 20:49, MAISONOBE Luc a écrit :

"Ward, Evan" <Evan.Ward@nrl.navy.mil> a écrit :

Hi,

Hi Evan,

I'm getting the following error when running `mvn test`. Is this an
actual failure or is the tolerance too tight? I'm not familiar with the
test so I figured I would ask here.

java.lang.AssertionError: expected:<0.687998> but was:<0.6880143632396981>
    at org.orekit.estimation.leastsquares.OrbitDeterminationTest.testW3B(OrbitDeterminationTest.java:310)

Yes the tolerance is too tight.

It is one example of the way we did validate some parts

 - first we run extensive tests and when we do not have reference
   data we change parameters to look how output changes, we check
   consistency with simulations, in some case we explore domain
   with random or Sobol drawings
 - then we convince ourselves that what Orekit computes is indeed
   correct
 - then we run a last time the test and pick up Orekit result itself
   as the reference
 - then we put a *very tight* and even non-physical tolerance so
   that when anything changes in Orekit the test crashes and
   developers at least notice the effect.

Then, if it seems the test crash is only due to the non-meaningful
tolerance, either we relax the tolerance or we even change the
reference value itself.


Also for the specific case of W3B, this was a satellite that experienced
a problem right after launch (a leak in the propulsion system), so the
satellite was intentionally put on a reentry orbit to burn in the atmosphere
rather than being sent to geostationary. So its dynamical model is really
particular (and interesting) and we can't have any really accurate orbit
on this. So I would say, just change the test so it pass again. Here, we
know the reference value is just what a previous Orekit version found and
the result is highly sensitive to changes in the code.

best regards,
Luc

Best Regards,
Evan