ESP_013948_1410, the kind of data we can't wait to get more of!

We are now returning to our normal mode of operations after several long months of being in safe mode. The anomaly on August 26th was the last in a series of computer glitches on board the MRO spacecraft that caused a reboot. The engineering teams have been working incredibly hard to get the anomaly figured out and prevent a possible side-effect from causing really serious problems. While they may not understand the original problem, and there is a chance it may happen again, they’re confident that at least it will not threaten the mission, so they’ve given us the go-ahead to resume normal operations.

Safe mode is a way of running the spacecraft where all of the science instruments are turned off and quiet. We still receive engineering telemetry so we can monitor temperatures and voltages. There are also “survival” heaters that prevent HiRISE from getting too cold in this mode.

During this time, the uplink operations staff has gotten a little restless. (more…)

Last week, the MRO spacecraft unexpectedly “safed.” This is when it reboots and puts itself into a precautionary mode; in this case it was in response to an unexpected voltage reading (more information in this press release about the safing). The engineers at JPL and Lockheed Martin spent long hours investigating the cause, making sure the spacecraft is healthy and unharmed (which it does appear to be), and cautiously getting things back to normal. I’ll jump right to the happy ending of the story, which is that we are now back to normal, imaging Mars as usual (here’s the press release about resuming routine operations).

What do the people at the HiRISE Operations Center (HiROC) do when the MRO spacecraft safes?
(more…)

Lately we’ve been working hard dealing with a LOT of extra data. Because Mars is getting closer to the Earth (you can visualize that in this view of the solar system), we are approaching the peak data rate for the entire primary mission. Not that we’re complaining! This just means the Targeting Specialists are planning many more images, and we’re making those images as big as we can.

Unfortunately, we can’t just make them all the largest size the instrument is capable of taking, because our camera will get too hot. If it overheats, the instrument will shut itself off in order to prevent any damage to the electronics. So we have to be careful, and only plan images that won’t overheat HiRISE. In order to predict those temperatures, we use a tool called HiTemp (of course!). Here’s what it looks like (click on the image to see a bigger version).

This program reads in our planning files, and then models the temperatures of two key spots on the focal plane of the camera. It’s our job to make sure we don’t go above the dotted red line – this gives us a comfortable buffer below the scary solid red line. That’s when HiRISE would shut off, or safe. We know from experience by now that this is a big pain in the neck – a lot of work is required to get us back up & running, and we miss observations while we’re turned off. So we watch our HiTemp plots!

Last week we had a rare event: HiRISE turned off! We call this safe mode, because it’s a safety measure built into the instrument’s software. Whenever any of the sensors starts going out of bounds, like temperatures or voltages, the instrument powers down to prevent damage to the electronics. In this case, one temperature sensor went over its upper limit of 35 degrees Celsius. It’s pretty disconcerting when something unexpected like this happens, but at least we know the instrument is protected.

We had the difficult detective job of figuring out what went wrong. It was clear early on that the instrument overheated, but we couldn’t figure out why. Our tool that predicts the temperatures (”HiTemp”) didn’t predict anything that hot. We didn’t take a really large image, which would heat us up (at least, nothing bigger than normal! ). The local operations team worked with the health & safety people, the spacecraft engineers at LMA, and some of the software developers at Ball Aerospace that originally designed HiRISE. Together we all investigated the problem.

(more…)