[Isocops] IBEX Time Stamp Issue: Follow-on Discussion

[NIgel Angold]

Regarding the possible paths forward, please see my comments in blue 
below. Your comments are welcome.

Thanks,
Nigel

  * Do nothing on the spacecraft
      o what are the consequences?
          + any timing issues on board?
              # It doesn't look like there are any at the moment but I'm
                concerned about the consequences of using memory
                locations outside the bounds of the table to determine
                spacecraft functionality
          + any ground timing issues e.g. MAESTRO archiving or SOC data
            processing?
              # MAESTRO is not able to archive data since everything has
                the same 1970 time stamp
              # If the secondary headertime is still incrementing, is it
                possible to update the MAESTRO processing to cope with
                the unexpected values?
      o not a comfortable option if we don't fully determine the root cause
  * Update on board table(s) or FSW
      o need to develop code fix, patch, procedures
          + several months to develop
              # Would this just be increasing the size of the table to
                include entries for another ~10 years?
      o need to test on Flatsat which has not been used for 5 or more years
          + NG is already working to get Flatsat up and running
              # Has NG been able to locate the password to gain access
                to the Flatsat machine?
      o could we poke memory locations rather than full table updates?
          + Are the memory locations beyond the timing table used for
            anything else? Could poking new values have unintended
            consequences?
  * Perform a Flight Computer Reset
      o hope that this corrects the memory that changed
          + I think this is probably our "Hail Mary" option
  * Set the on board spacecraft time to current time minus 30 years
      o this "Back to the Future" approach would put the time into the
        middle of the on board table (it starts at 1980)
      o would result in unique time stamps for archive purposes (unless
        the mission lasts more than 30 years)
      o would need to update MPS code to subtract 30 years from ATS
        command times or manually update each ATS file
      o could perform all MPS planning with current date so that flight
        dynamics processes would not be impacted
          + This may work but I'd like to make sure that going back in
            time results in MAESTRO producing good secondary header time
            stamps again
  * Another Path?
      o What have we missed?

One hypothesis is that the content of the memory just beyond the GPS 
timing table changed on 24 August. However, Ken points out that it's 
possible the content didn't change but the value is such that the time 
comparison wasn't exceeded until 24 August, thereafter resulting in 
further comparisons that were exceeded for another 6 locations until the 
location corresponding to year 2026 contained a value greater than the 
current time. If that's the case, then setting the spacecraft time back 
to just before the 24 August time stamp jump would result in times that 
MAESTRO could again process correctly, followed by another jump back to 
1970. That's a test that I think I'd like to try since I'd really like 
to know if this issue is caused by (random) memory content changes or 
simply because a static memory value has been exceeded.

Ryan points out that we'd need to stop the on board ATS if we did a 
"back in time" test. The orbit 540 ATS runs until 2021/09/14 08:30:00 
and the orbit 541 ATS starts at 2021/09:14 14:43:12. We could add a 
contact at around 2021/09/14 13:00-14:00 during which we'd delete the 
orbit 541 commands, set the spacecraft time to 24 August, watch for the 
time stamp jump, reset the spacecraft clock to NOW and reload the orbit 
541 ATS (and orbit 542 ATS, for that matter). How does that sound? Can 
you think of any unintended consequences? Can we plan this for next 
Tuesday at ~09:00 EDT without stressing the system?

Feedback please!

Thanks as always,
Nigel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.sr.unh.edu/pipermail/isocops/attachments/20210908/11ce4c83/attachment.html>