What to do When Problems Occur

If you should run into problems during the night, first look through the online manual and the Troubleshooting section below for possible solutions. If you still can not solve the problem, and it's before 10:00, call the Site Manager for assistance - phone numbers are on the white board. If it is after 10:00 and you are unable to continue observing, and have exhausted all other possibilities feel free to call the Site Manager. Do Not Call Kitt Peak Electronic Maintenance, or the WIYN operator unless told to do so.

Telescope problems
TCS problems
Dome problems
S2KB problems
[Mosaic] Guider problems
Dry Air System
Dewar Warmups
S2KB Guider problems
Mosaic problems
IRAF problems
Computer problems
Facility issues

Troubleshooting Tips

Telescope problems

• There are star trails in images
  • There are several possible causes for star trails in your images. Try the following in this order:
    • The telescope takes several seconds to settle down after any move - offsets or slews. Be sure to wait 10-15 seconds after moving the telescope before beginning an exposure.
    • Check to see if your preload motors are on. Located in the computer room the power supply for the preload motors is housed in the first set of racks as you walk in the door (same one that contains your flat field lamps). The second rack from the bottom is your preloads power supply. First check the front panel to see if there is voltage. If not, you will need to turn this on. On the back of this rack there is a small toggle switch. Put the switch in the On position. If the switch is already on and there is still no voltage, contact the Site Manager.
    • There is a spot in the sky where the telescope is known not to track well. This spot is in the northwest. If you suspect this is the problem you can try turning off the preloads to see if it helps. On the ACE TCS select Telescope -> Preload Motors -> HA preload (to turn the HA preload off). Then select Telescope -> Preload Motors -> DEC preload (to turn the DEC preload off). Remember to turn the preload motors back on when moving away from this area.
    • If the stars trail North/South and you are using S2KB there may be a problem with the CCD shutter. Check the section titled "S2KB problems" below.
• The telescope is in a soft limit (one of the limits boxes is red on the TCS).
  • Note which limit you have run into. On the TCS the box that is red should read either E, W, N or S.
  • Visually inspect the telescope to make sure no damage has occurred by running into the limit.
  • Move the telescope in the opposite direction with the hand paddle.
    • Go upstairs and remove the hand paddle from it's stowed position near the liquid nitrogen fill line (stowed on the chain railing).
    • The toggle switch on the top of the hand paddle controls the speed at which you move the telescope - "slew" speed (fastest) is engaged when pushing the toggle switch to the left, "guide" speed (slowest) when the toggle switch is in the center position, and "set" speed (medium) when the toggle switch is pushed to the right. Push the toggle switch to the left to use "slew" speed. Note that while near the limit slew speed is reduced to set speed, but when clear of the limit the speed increases to nominal slew speed.
    • Push the button related to the opposite direction from your limit (e.g. if you have hit a West limit push the East button).
    • Move the telescope well out of the limit area.
  • Stow the hand paddle back on the railing and resume operations from the TCS.
• The telescope is in a hard limit (3 degrees above horizon)
  • You must contact the Site Manager to recover from this
• Telescope wont move in either RA, Dec or both
  • First check to see if the safety interlock is visible in the TCS.
    • Clear the interlock by clicking the reset button.
    • If the interlock will not clear see instructions below, under TCS problems, for what to do when interlock will not clear
  • Are you in a limit? See instructions above.
  • There is a way to reset the drives via the software on Olive
  • On the desktop there is a Dynaserver icon that says Dynaserver RA=3 Dec=4
  • When the GUI is opened there is a pull down menu on the top left for picking what you want online. RA is 3, Dec is 4. Choose the appropriate drive.
  • Click the connect button on the upper right of the GUI and wait for the connection to complete. There will be two green buttons on the upper right of the GUI. You may have to try connecting a couple times for it to work.
  • Under menu, under operation, click the drive button. A message box may pop up that says: Operation authorith is swithced to the serial I/F side. Is this ok? Click yes.
  • When the new GUI opens up click on error reset. When error reset button goes gray the drive has been reset.
  • There is a button on the right hand side that says either controller side, or I/F side. It MUST say I/F side when you exit or the telescope will still not move.
  • Close the error reset GUI
  • Click disconnect and wait for the disconnection
  • Repeat for other drive if necessary
  • Close Dynaserver and resume telescope operations
  • Occasionally the motor breaker may have to be reset, but this is rare.

TCS problems

• When sending the focus to a specified number the focus takes off.
  • Hit the Stop button on the focus inspector GUI and try again.
  • If it continues to take off select the "Go to Zero" button on the focus GUI. This will reinitialize the focus motor by taking to a limit.
  • Gradually back it out of the limit by moving in small steps (+30 units) until the focus readout is no longer red.
  • Be Warned: the focus may be drastically off from what you expect after this. Recheck focus before resuming observing.
• Focus is stuck at a limit
  • Click the STOP button on the Focus Inspector GUI
  • Click the Drive to Zero or Drive to In button to drive the focus to a limit.
  • If the focus still does not move you will need to reset the Telescope Control Crate in the Computer room by unplugging the crate and plugging it back in again. Only do this if you have been shown how!
  • Be Warned: the focus may be drastically off from what you expect after this. Recheck focus before resuming observing.
• The focus runs away when changing filters(S2KB only).
  • Make sure the "Change with filters" box is not selected on the focus GUI.
  • To recover from a focus runaway see above.
  • Be Warned: the focus may be drastically off from what you expect after this. Recheck focus before resuming observing
• Can't enter a specified declination
  • There is a known bug that will not allow you to enter in only the degrees for Declination - when hitting the GoTo button the units you have entered will have been overwritten.
  • You must also type in the minutes and seconds (instead of 75, enter 75 00 00).
• Problems with the TCS/Instrument Communication.
  • See the Troubleshooting Flow Chart
• The Filter Wheel is not responding.
  If the filter wheel is grayed out and will not move to the desired position, try the following steps in order.
  1. In the filter wheel GUI check which filter wheel is grayed out (wheel 1 or wheel 2). On the right hand side of the filter wheel GUI click INIT Wheel 1 or INIT Wheel 2, depending on which wheel is not responding.
  2. Go upstairs and move the faulty wheel by hand to check for any mechanical obstructions (if you don't know how to do this call the Site Manager). If it does not move, stop and call the Site Manager immediately.
  3. If the wheel moves freely Exit the ACE TCS (under User select Exit). Then restart the ACE TCS via the icon on the desktop - you will also need to restart the ACE socket server (under Network select Start Socket Server).
• The platform interlock gui pops up and will not clear.
  • Quit out of ACE
  • From the Windows start menu select shutdown computer, then select reboot
  • When computer reboots, log into ACE and resume observations
• When moving to a new target the status bar tells you "Dec Locking" and it never locks
  • The telescope HAS reached its target in Dec
  • Make sure the telescope has reached its position in RA and hit the "Cancel Slew" button
  • Start observing

Dome problems

• The dome runs away
  • Select Dome => Stop from the pulldown menu or click the dome stop button on the toolbar.
• The dome position is lost
  • Select Dome => Home from the pulldown menu. This will rotate the dome until it finds its home position.
    This may take a few minutes so be patient while the dome moves.
  • If the dome never seems to find home you may need to reset the dome home position manually. To do this you need to go upstairs and align two sets of duct tape - turn off Dome tracking first. The dome home position is at 90 degrees, which is close to the dome Flat Field position of 73. From the control panel on the platform, move the dome either Left or Right to the approximate dome flat field position. There are two strips of duct tape low on the wall, left of the dome vent control. There are two matching strips of duct tape on the dome wall that you will need to align with them. If the dome is approximately in the flat field position you will need to move the dome ~20 degrees to the right. If you have aligned the pieces of tape well the dome will have reset this position as 89 degrees. Go downstairs and look at the dome azimuth readout. It should now read 89.0 HOME. If not you can tell the TCS that the dome is at 90 degrees. On the pull down menus select Dome => Reset Encoders. Enter 90 in the box and click RESET.
  • If the above procedures do not fix the positioning problem you may need to reset the dome encoder a different way. To do so:
    • Take hand paddle and move the dome due south.
    • Stand at the far north point of the dome so you are under the polar axle.
    • Check that the dome slit is bisected by the polar axle. If not move dome until that condition prevails. The amount you moved was the error in the system.
    • Then in the TCS pull down menu select Dome => Reset Encoders. Enter 180 in the box and click RESET.
• I've turned Dome tracking on and the dome moved, but not to the correct position.
  • Toggle the dome tracking off and then on again.
    • Select Dome => Smart Dome. Acknowledge turning the tracking off.
    • Select Dome => Smart Dome. Acknowledge turning the tracking on.
  • If the dome is still lost see the above procedure for resetting the dome position.
• The dome is making a lot of noise but is not moving. Sometimes the dome gets stuck in one position. If this happens:
  • Turn dome tracking off.
  • From the control panel upstairs move the dome in the opposite direction close to the desired position.
  • Turn dome tracking back on from the ACE TCS
• The dome status says sleeping and the dome will not track
  • Most likely you have adjusted the dome pointing while AutoDome was on.
  • Toggle the dome tracking off and then on again.
    • Select Dome => Smart Dome. Acknowledge turning the tracking off.
    • Select Dome => Smart Dome. Acknowledge turning the tracking on.

S2KB problems

• "Error on line 47" or "Floating Operand Error" in Data Acquisition window and you can't take any exposures
  • Is the socket connection up? This needs to be up to be able to take any exposures.
• The Data Acquisition window hangs when trying to start an exposure
  • Is the socket connection up?
• Stars trail North/South and the tracking is on:
  • The shutter may be stuck open.
  • On Olive select Instruments => ACE Linear Shutter => Reboot.
• Data Acquisition window says "Preparing..." and the exposure never starts.
  • This is a new "feature" that has been introduced into the system in September 2004, and is usually seen when typing anything except "observe" in the Data Acquisition window. The exact cause of the problem is unknown, but seems to be easy to recover from.
  • Check whether the socket connection is up and that only numbers (no words) are scrolling through the xterm window with the VNC viewer window.
  • Type Ctrl-C, then flpr. This should clear the hang and you should again be able to start an exposure.
• An exposure looks like a bias image.
  • Is the mirror cover removed and the dome shutter open?
  • Is the dome in the way?
  • Is it cloudy?
  • The shutter may be stuck closed. See the instructions above for resetting the shutter.
• When displaying an image only the central 1k x 1k is displayed.
  • In the Data Reduction window type: set stdimage=imt2048
• The images no longer display automatically in ximtool
  • type epar obspars in the data acquisition window
  • add the line postproc %s & to the postproc command line
• Taupe VNC is no longer responsive
  • Reboot Taupe
    • From a browser window go to the URL http://36-power-1
      • Login and password are on the whiteboard
    • Click on cycle for all 4 Taupe disks
    • Logout of 36-power-1
  • NOTE: Taupe takes many minutes to reboot. Be patient!

Dry Air System

• Dry air power is on, but no pressure reading on either gauge
• Main Compressor has probably failed.
• Turn off dry air system power
• Unplug main compressor and autodrain in exterior room
• Plug-in and turn on black back-up compressor and autodrain in computer room, wait until compressor has built up about 60psi.
• Swap pink for black air inlet hose on incoming side of dry air system
• Turn on dry air system
• Make sure to turn off system and unplug compressor and autodrain at end of night
• Dry air box does not seem to be working.
• Switch to use of nitrogen tanks.
  • Turn power switch on dry air system to off
  • Flip valve to N2
  • Open valve on top of N2 tank

Dewar Warmups

• If your CCD and/or dewar has been allowed to warm up above nominal temperatures you will need to fill the dewar immediately. Note: there are 2 distinct temperatures - one for the CCDs and one for the dewar. Your Dewar temperature will start to increase before your CCD temperature. If either of these has started to warm, stop and refill your dewar.
  • Stop observing and move the telescope to the Zenith Park position.
  • If using Mosaic, alert the WIYN OA that the dewar has warmed up and ask them to please refill the dewar.
  • If using S2KB, follow the normal procedure for filling the dewar and note on the logsheet that a warmup has occured.
  • In both instances, the WIYN 0.9m Site Manager should be notified (via email is sufficient - hmathis at noao.edu).
  • If the CCD temperature is also above nominal operating temperature, you will need to wait until it is back to nominal temperature before taking science data.
  • Because the dewar is significantly warm, it will burn off the liquid nitrogen rather quickly. Due to this you will need to refill the dewar in another 2 hours.
  • Keep a close eye on the dewar temperature for the next 8 hours. If it begins to warm up, again stop and refill your dewar. If you're sure the dewar has been getting a complete fill and it looks like the hold time has decreased, notify the 0.9m Site Manager (hmathis at noao.edu).
  • To check your CCD and Dewar temperatures when using S2KB, in your Data Acquisition window type ccdinfo. The temperatures will be displayed at the bottom. When using Mosaic, the CCD and Dewar temperatures are continously displayed in the Mosaic GUI.
• For hints on properly filling a dewar see the section in the Appendix Dewar Filling Tips and Tricks.

S2KB Guider problems

• Stars trail or bleed when "expose" is selected, but not when "track" is selected.
  • Our guide cameras are small CCDs. We do not operate the shutter on these CCDs under normal guiding conditions and so you may see star trails in you images due to the shutter being open upon readout.
• No stars are seen on either guide camera and there should be lots of guide stars in this field.
  • Is the mirror cover off and the dome shutter open?
  • Is it cloudy?
  • Are you pointed at the side of the dome?
  • Sometimes this is due to the temperature of the guide camera being too warm. On the Setup tab in the CCD control window within MaximDL check what your setpoint temperature and your current temperature are for the currently connected camera. If the temperature is well above the setpoint temperature the stars will be "drowned out" by noise on the camera. Wait until the camera comes down to nominal temperature (takes only a couple minutes) and try again.
  • Also, the guider software automatically subtracts a dark from the guide camera image. Another way you can solve this issue is to tell the software to not subtract a dark frame from the image. To do this select the Settings button on the Guide tab and uncheck the box at the top next to Autodark.

[Mosaic] Guider problems

• The numbers in the "corrections" boxes on the TCS aren't changing, ie. the connection between Moss and Olive is down.
  • Have you turned the guider on in both places? Be sure to turn it on on Moss (via the GUI on Emerald) and Olive?
• The video on Moss is hung.
  • Try restarting the guider software by selecting "Restart Guider" from the Mosaic Menu. If it is still hung you will need to reboot Moss. Locate Moss's CPU in the computer room. Pulldown the front cover and push the red reset button. Wait for Moss to reboot into Linux. You'll need to login again - name and password are on the whiteboard. After loggin in restart the guider software on Emerald by selecting "Restart Guider" from the Mosaic Menu.
• The Guider GUI never opens on Emerald.
  • In an xterm window on Emerald, type "xhost moss". Select "Restart Guider GUI" from the Mosaic Menu.
• The Guider video window crashed and disappeared.
  • Close the Guider GUI window and Guider video window.
  • Select "Restart Guider" from the Mosaic Menu.
  • Can't see anything on either Mosaic guider cameras.
    • Is the shutter set to "guide" on the Mosaic GUI?
    • Are the cameras on (the intensity and power)?
    • Is the ND filter in?
    • Is the dome shutter open and the mirror cover removed?
    • Is the dome in the way?
    • Is it cloudy?
  • The box on the Mosaic guider will not move when selecting "Warp to Star".
    • The "Warp to Star" command sends the boxes to the brightest star in the field. If the brightest star is on the edge of the field the box will not move there and you will need to select another star manually:
      • Move the cursor to another star and left-click the mouse.
      • Or jog the telescope, via the softpad on the TCS, enough to bring the star into the field. Click warp to star again.
  • The telescope "jumps" when turning the Mosaic guider on
    • Exit and restart the ACE TCS program on Olive. You will need to turn your telescope and dome tracking back on.

  IRAF problems

  • When displaying an image only the central 1k x 1k is displayed.
    • In the Data Reduction window type: set stdimage=imt2048
  • A graphical IRAF window has poor colormapping.
    • In the Data Reduction window type cl> reset stdgraph=xgtermb. This will reset the window into black/white.
  • Get the error message "Can not open device _" when trying to allocate a tape device.
    • This will require a hard-boot of the data taking machine. Call the Site Manager for help on this.
  • How to change between .imh and .fits for your data files:
    • In your Data Acquisition window on Taupe type set imtype=fits or set imtype=imh.
    • Alternatively you can edit your login.cl file. In the middle of the file there is a section with several #set PARAMETER=VALUE. Edit the last line in this section, which reads #set imtype = "imh" [or "fits"]. Remove the # at the beginning of the line, and input the desired file type (either imh or fits) at the end of the line.
    • You will need to restart your Data Acquisition window before the change taks affect.
  • Can not write to tape using IRAF.
    • You may need to specifically set the blocksize to zero.
      To check the blocksize type mt -f /dev/nst* status
      To set the blocksize type mt -f /dev/nst* setblk 0
  • Other IRAF specific questions can be directed to the IRAF group at iraf at noao.edu.

  Computer Problems
• For problems with Olive please see the TCS problems section
• For problems with Moss please see either the S2KB or Mosaic guider problems section
• For problems with Taupe please see the S2KB problems section
• For problems with Mosaic please see the Mosaic Manual

Facility issues

• Running out of drinking water in the control room.
  • Note that even though you don't see water in the clear bottle, the bottom, blue container is a reservoir and most likely contains water. The water level is checked several times per week and refilled as needed. If the bottle and reservoir are indeed dry send an email notification to the Site Manager at fhaase@noao.edu.
• Running out of liquid Nitrogen.
  • The dewar scale reads 348lbs when the dewar is empty. The liquid nitrogen tank level is checked and filled on Mondays, Wednesdays and Fridays as needed. If the level is below 1/4 tank when you arrive at the telescope in the afternoon, notify the site manager by email fhaase@noao.edu or by phone (phone numbers are on the whiteboard in the control room).
• Unusually low pressure from the liquid nitrogen line.
  • Sometimes you may see very low pressure on the liquid nitrogen line when filling the dewar. This is a rare occurance, but will most often occur shortly after the liquid nitrogen storage tank has been refilled. If there is a lot of pressure in the tank the pressure will bleed off through the overpressure valve, eventually freezing the valve open. The valve will eventually warm up, close and the tank will build pressure again, but this may take a couple hours to get the pressure up to the normal 22psi.
  • It is safe to fill the dewar even if the pressure is lower than normal. However, it may take significantly longer to fill the dewar.
  • Note: It is normal to hear bleed off from the storage tank, especially after filling the dewar or when the tank has been refilled. This is ok.

Last updated June 1, 2012