Design Goals:

LESSON LEARNED: Pick one programming language to work in…  and make it Python.   It is absolutely essential for astronomers to work closely with professional software developers on the telescope control software. It’s easy to find developers who can work in Python. It’s pretty much impossible to find astronomers who can work in Java.

We divide M&C into three main areas:

Telescope Configuration:

LESSON LEARNED: Wherever possible, give every digital hardware component a unique serial number, and every software component a version number. Pass all those serial numbers and version numbers along in the data stream, and record them with the data!   I cannot stress this enough. Digital serial number chips cost a few cents, and embedding this data in packet headers or a side-channel of metadata is a negligible cost compared to the total data traffic.

Telescope State:

LESSON LEARNED:  Part of the process of requesting an observation should be defining ‘minimum useful’ thresholds for that observation, in terms of number of elements working, RFI level, etc, because a 100% functional telescope will never exist. This can be used in the longer term (to decide when to schedule a particular observing block) and in real time (to decide whether to actually record data or not, given the current observation and telescope state).

Telescope Telemetry:

LESSON LEARNED: Decide early how to manage these three classes of telemetry (SQL database, round-robin database, text files, etc), but be prepared for measurements to change category after a few years. You don’t want to have to scrape the data from years of archived log files (in different formats) when someone decides that it’s important…

LESSON LEARNED: If you store telemetry data in a text file, don’t just write it to a log file embedded in the usual error/debugging messages. Use a human readable log file for errors and debugging, and a CSV file (or similar) for telemetry data.

LESSON LEARNED: However the telemetry data is stored, build the interface for visualising that data at the same time you start storing it. Storing data that you can’t visualise or use is pointless.

LESSON LEARNED: Give every discrete hardware unit at least one bright externally visible power LED, and if it contains a processor, flash an LED in the main loop to show that the software is running. Technicians need a quick and foolproof way to tell if a box is powered up and working, powered up but not working, or turned off, without opening the (RFI-tight, dozen-screw) box. Having software-controlled lights on the boxes we have out in the desert has probably already saved as much money in technician time as the capital cost of the hardware.

LESSON LEARNED: Software controlled lights are essential for internal rack-mounted gear too. There needs to be a way for a human to quickly verify that the physical locations and wiring of equipment (a station of TPM's, for example) matches the internal M&C representation of that hardware. Something as simple as a light-chaser scrolling across the LED's of all of the TPMs in sequence would make a mis-identified bit of hardware stand out to the eye. Software controlled lights are also incredibly useful to identify faulty units that need to be swapped out, so a good unit isn't replaced by accident.

In Summary, looking ahead to the SKA: