There are multiple tables of data necessary in SEMS. These will primarily live on site, but will be replicated back to SBA.
The different tables are:
- Internal instrument data: Data recorded from LCOGT-run instruments.
- External environment data: Data read from external instruments, websites, or feeds that has been used by LCOGT.
- Event logging: Explanation of a site change of status, what factors led to the decision.
Internal Instrument Data
This data will be recorded at a 12-second rate. These records are actually the product of 4-second readings in the CR1000. It is assumed these value are reliable because the CR1000 handles averaging over the 12 second window, eliminating the need to further filter and throw out spurious values.
Data Compression
To reduce data size, older Internal Instrument Data will be continuously compressed into more coarse time samples. Current data will be kept at the 12-second rate, while the oldest data will be condensed to a 20-minute rate. The compression level of the data will be stored with each record, defaulting to zero with the original 12-second rate. This is the proposed compression scheme:
Data Compression Scheme
| Level | Sample Rate | Lifetime |
|---|
| 0 | 12 seconds | 7 days |
|---|
| 1 | 120 seconds (2 minutes) | 2 years (730 days) |
|---|
| 2 | 1200 seconds (20 minutes) | infinite |
|---|
This scheme will produce the following number of records:
Data Size with Age
| Year | # Records (approx) |
|---|
| 1 |
300000 |
| 2 |
576000 |
| 5 |
654840 |
| 10 |
786240 |
| 20 |
1049040 |
External Environment Data
Event Logging
Replication to SBA
SEMS hosts a MySQL database on site to function in the absence of an internet connection. However, this database is replicated to Santa Barbara for long-term storage. The local (on-site) database is regularly purged of data older than one week. The compression algorithm (see above) runs daily on the data stored at Santa Barbara.
