[Isocops] EXT :IBEX Time Stamp Issue: Follow-on Discussion
NIgel Angold
nangold at princeton.edu
Thu Sep 9 15:13:19 EDT 2021
Thanks Ryan.
So I think the next step is to create a memory map or perform a RAM
read. When do you think we can have a CAR ready for review?
Thanks,
Nigel
> I’ll add my comments below in red.
>
> -Ryan
>
> *From:*NIgel Angold <nangold at princeton.edu>
> *Sent:* Wednesday, September 8, 2021 10:50 PM
> *To:* Perry, Timothy E [US] (SP) <Timothy.Perry at ngc.com>; Wynn, Reese
> [US] (SP) <Reese.Wynn at ngc.com>; Bobbett, James E [US] (SP)
> <James.Bobbett at ngc.com>; ISOC at New Hampshire
> <isocops at lists.sr.unh.edu>; Bachir-Bouiadjra, Benahmed [US] (SP)
> <Benahmed.Bachir-Bouiadjra at ngc.com>; Conway, Jim [US] (SP)
> <Jim.Conway at ngc.com>; Leonard, Benjamin Y [US] (SP)
> <Benjamin.Leonard at ngc.com>; Ken Fairchild UNH (Ken.Fairchild at unh.edu)
> <Ken.Fairchild at unh.edu>; Dave McComas <dmccomas at princeton.edu>; Bret
> Hautamaki <bhautama at gmail.com>; Fleming, Ed [US] (SP)
> <Ed.Fleming at ngc.com>; Wesley, Sheral R [US] (SP)
> <Sheral.Wesley at ngc.com>; Tyler, Ryan S [US] (SP) <Ryan.Tyler at ngc.com>;
> Ibex Mission Operations <ibexops at gmail.com>; Ken Fairchild
> (kwfairchild at gmail.com) <kwfairchild at gmail.com>; Eric Christian
> <eric.r.christian at nasa.gov>; Cavallo, John A [US] (SP)
> <John.Cavallo at ngc.com>; Chubin, Eric [US] (SP) <Eric.Chubin at ngc.com>;
> Conway, Jim [US] (SP) <Jim.Conway at ngc.com>; Cross, Walker W [US] (SP)
> <Walker.Cross at ngc.com>; Varia, Apurva P. (GSFC-5840)
> <apurva.p.varia at nasa.gov>; IBEX FDG Group (ADS-IBEX at l3harris.com)
> <ADS-IBEX at l3harris.com>; Scott Weidner <sweidner at princeton.edu>
> *Subject:* EXT :IBEX Time Stamp Issue: Follow-on Discussion
>
> Regarding the possible paths forward, please see my comments in blue
> below. Your comments are welcome.
>
> Thanks,
> Nigel
>
> * Do nothing on the spacecraft
> o what are the consequences?
> + any timing issues on board?
> # It doesn't look like there are any at the moment but
> I'm concerned about the consequences of using memory
> locations outside the bounds of the table to determine
> spacecraft functionality
> # I agree it doesn’t look like there are, and indeed
> that’s been true for 1.5 years before our recent
> speedbump. Once we confirm the contents of memory
> where the table is looking, I might be satisfied.
> + any ground timing issues e.g. MAESTRO archiving or SOC
> data processing?
> # MAESTRO is not able to archive data since everything
> has the same 1970 time stamp
> # If the secondary header time is still incrementing, is
> it possible to update the MAESTRO processing to cope
> with the unexpected values?
> # I’m wondering if we can edit the binary files to fix
> the secondary headers based on a known conversion so
> that with a little (repeatable) tinkering the
> downlinked files can make it into the archive after all
> o not a comfortable option if we don't fully determine the root
> cause
> * Update on board table(s) or FSW
> o need to develop code fix, patch, procedures
> + several months to develop
> # Would this just be increasing the size of the table to
> include entries for another ~10 years?
> # 2 major options:
> * Update several rows in the table to match upcoming
> years and update code to make the starting year
> 2020 rather than 1980 – should fix all aspects of
> the issue, and we’ve already tested this logic
> * Simple fix to edit only the table value in the
> last row of the table (2020) to make it a very
> large number that the GPS time will never exceed.
> This should lock in a 2019 year and correctly
> calculated number of days that can easily be
> converted (as MAESTRO had been doing automatically
> since the start of 2020).
> # Neither of these options involve changing the table
> length and both should be achievable via RAM poke
> # We need some more time to do our research on this,
> figure out exactly what memory locations we’re talking
> about.
> o need to test on Flatsat which has not been used for 5 or more
> years
> + A poke wouldn’t necessarily need to be tested on FlatSat.
> We’d start by reading the RAM at that location to verify
> we’re touching the right part of memory, abort if not
> confirmed. Then we poke just that same location. Since
> this is a RAM poke, we can always reboot if something goes
> wrong and come up cleanly (in 1980, a time that would be
> correctly interpreted), clearing out the poked RAM.
> + NG is already working to get Flatsat up and running
> # Has NG been able to locate the password to gain access
> to the Flatsat machine?
> o could we poke memory locations rather than full table updates?
> + Are the memory locations beyond the timing table used for
> anything else? Could poking new values have unintended
> consequences?
> + I definitely would not want to poke beyond the table
> (thus, no extending the length of the table as an option)
> * Perform a Flight Computer Reset
> o hope that this corrects the memory that changed
> + I think this is probably our "Hail Mary" option
> + Only really makes sense if the test you propose below
> shows that now timestamps from a clock set to 3 weeks ago
> (prior to the recent speedbump) also turn out bad.
> * Set the on board spacecraft time to current time minus 30 years
> o this "Back to the Future" approach would put the time into the
> middle of the on board table (it starts at 1980)
> o would result in unique time stamps for archive purposes
> (unless the mission lasts more than 30 years)
> o would need to update MPS code to subtract 30 years from ATS
> command times or manually update each ATS file
> o could perform all MPS planning with current date so that
> flight dynamics processes would not be impacted
> + This may work but I'd like to make sure that going back in
> time results in MAESTRO producing good secondary header
> time stamps again
> * Another Path?
> o What have we missed?
>
> One hypothesis is that the content of the memory just beyond the GPS
> timing table changed on 24 August. However, Ken points out that it's
> possible the content didn't change but the value is such that the time
> comparison wasn't exceeded until 24 August, thereafter resulting in
> further comparisons that were exceeded for another 6 locations until
> the location corresponding to year 2026 contained a value greater than
> the current time. If that's the case, then setting the spacecraft time
> back to just before the 24 August time stamp jump would result in
> times that MAESTRO could again process correctly, followed by another
> jump back to 1970. That's a test that I think I'd like to try since
> I'd really like to know if this issue is caused by (random) memory
> content changes or simply because a static memory value has been exceeded.
>
> Ryan points out that we'd need to stop the on board ATS if we did a
> "back in time" test. The orbit 540 ATS runs until 2021/09/14 08:30:00
> and the orbit 541 ATS starts at 2021/09:14 14:43:12. We could add a
> contact at around 2021/09/14 13:00-14:00 during which we'd delete the
> orbit 541 commands, set the spacecraft time to 24 August, watch for
> the time stamp jump, reset the spacecraft clock to NOW and reload the
> orbit 541 ATS (and orbit 542 ATS, for that matter). How does that
> sound? Can you think of any unintended consequences? Can we plan this
> for next Tuesday at ~09:00 EDT without stressing the system?
>
> I’m in favor of such a test, but it might prove unnecessary if we can
> establish via memory map (and possibly a RAM read command) that Ken’s
> hypothesis is correct.
>
>
>
> Feedback please!
>
> Thanks as always,
> Nigel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.sr.unh.edu/pipermail/isocops/attachments/20210909/42ed90dc/attachment-0001.html>
More information about the Isocops
mailing list