ShaunG

This isn't a matter of "3 kHz not being a very high frequency" though. If you were measuring a 3 kHz signal from a non-digital based source such as an op-amp oscillator you might not be having the same issues. Digitally synthesized signals have lots of interesting things going on in them. If you have access to a spectrum analyzer try looking at your signal with it to see how "clean" it really is. You'll see your signal, harmonics of your signal, possibly even reference signal frequencies used inside the hardware and their harmonics, etc.

When you increase the sampling rate you are starting to measure various higher order frequency harmonics associated with the DAC signal generation, ADC signal measurement, etc. These are some of the causes of the glitches you see when measuring with an ADC. By decreasing the sampling rate you aren't able to properly sample/measure the signals with higher frequencies. I had a hunch something like this was going on when every 3rd wave looked bad. What exactly are you trying to measure? The waveform shape, RMS voltage, or both? If you want to measure a clean waveform shape try a higher sampling rate that isn't an integer frequency harmonic of the waveform you are sampling to see if it helps.

Try using a lower sampling rate. Start at 20 KS/sec , and increase the rate by 3 KS/sec. Also, is it every 3rd sample peak that is distorted to ~1V?

You say your sine waves are "not always nice", but have you looked at your sine wave output on an oscilloscope to independently verify what they look like? If the Keithley DMM is measuring the expected value, then I would think that the output should appear "clean" (not like the picture above) on the scope. Near the zero crossing it's not unexpected to see problems in waveform measurements simply due to the resolution limit of the measuring device. That doesn't explain what is happening on some of the waves which approach ~1V when they shouldn't. That is what is causing your calculated RMS value to be higher than expected, so fix that and you'll fix your problem... One thing I would do to see if it helped is change the voltage range used on both devices from -10V to 10V to something more in the range of -1V to 1V or -5V to 5V.