Previous Article Next Article Table of Contents
Trillions and Trillions Saved (1Mar96) (from KPNO, NOAO Newsletter No. 45, March 1996) The NOAO Save the Bits archive passed the one Terabyte milestone on 4 December 1995 at 23:47 MST. The lucky observer was Brian Rachford (Wyoming) who was observing an H II region at the Coude Feed with a grism cross disperser, camera 5 and F3KB. KPNO archiving began on 20 July 1993 with the pilot queue observing program at the 2.1-m telescope. Data from 8 separate telescopes and 12 different data acquisition computers have been multiplexed over the network onto a single archive medium in the Kitt Peak Administration building. Dual exabyte 8505 format copies are recorded of data from KPNO, WIYN, and NSO nighttime instruments. FITS header information is separately recorded and cross referenced to the data tapes allowing a simple retrieval program to access any archived image within about ten minutes. Image statistics for 1995 are provided in the table with the 1994 and grand totals for comparison. The addition of WIYN data to the archive data stream (partially offset by the removal of 1.3-m and nighttime McMath-Pierce data) should raise the science total for 1996 alone past the 1/2 Terabyte level. A few calculations may help visualize the size of this number: (1) 1/2 Terabyte = 1/4 trillion pixels = one image 500,000 pixels square. (2) A 500,000 square CCD chip with 21 um pixels would be 10 meters on a side. (3) If it takes 2 minutes to read out a 2048 x 2048 CCD at a nominal gain through one amplifier, it would take 80 days to read out a CCD this size. (4) 1/2 Terabyte/year = 16 Kilobytes/second. The archive could keep eleven 14400 baud modems busy 24 hours a day, 365 days a year. Over the last two years, about two dozen requests for archive data have been received. The majority of these come from observers who have suffered data disasters of various sorts and need to recover the original data from their observing runs. A growing fraction of the requests support science that is independent of the original observing proposal (at the Director's or original observer's discretion). Save the Bits is an ongoing project. The software is simple to install and is available to outside sites (e-mail rseaman@noao.edu for details). The W.M. Keck Observatory has been relying on their own Save the Bits installation since February 1995. An installation is pending shortly at CTIO. The next phase of the project at KPNO is to ingest the FITS header catalog (currently itself about 2 Gigabytes in size) into a relational database with a Web browser front end (using the WDB interface from CADC/ESO). NOAO Archive Statistics by Telescope for 1995 Telescope Total Images Object Exposures Calibrations Number Mbytes Number Mbytes Number Mbytes Mayall 4-m 58,530 64,982 49,559 31,843 8,971 33,141 WIYN 11,854 70,960 5,211 22,939 6,643 48,022 2.1-meter 71,802 43,283 54,352 23,108 17,450 20,177 Coude Feed 27,223 28,454 12,871 11,932 14,352 16,525 McMath 25,364 2,622 10,005 1,199 15,359 1,421 *1.3-meter 20,842 5,308 18,436 4,687 2,406 621 0.9-meter 21,843 124,323 12,201 66,906 9,642 57,415 Schmidt 23,520 129,720 13,813 64,008 9,707 65,713 1995 Total 260,978 469,652 176,448 226,622 84,530 243,035 1994 Tota 275,850 442,023 173,903 223,121 101,947 218,915 Nightly 650 1,166 413 567 237 599 Total to Date 581,584 1,042,040 369,059 506,662 212,525 535,397 * The Kitt Peak 1.3-m IR telescope was retired in May 1995. Final data were archived on 28 March 1995. o Nightly and total to date represent 894 calendar days between 20 July 1993 and 31 December 1995. o WIYN archiving began officially on 9 August 1995. o About 45,000 GOES weather pictures and 10,000 WIYN commissioning images are not shown here. o By number, 63% of the images are object exposures. By data volume in Mbytes, 49% are objects. o IR data are about 50% of the exposures, but 5% of the data volume. o The smallest image archived is 0.01 Mbytes, the largest 72.52 Mbytes. o The 1,000,000,000,000th byte was archived at 11:47 pm on 4 December 1995. Rob Seaman
Previous Article Next Article Table of Contents