I think that the optimal situation is to use all the dynamic range(16777216 or 65536 levels) in PCM, if it is too loud, i prefer using the analog volume knob or modify the gain in circuit.
now i have a true 24 bit file(quite sure it is 24bit recording since i DIYed a NE5532 2ch mic preamp, 2 mics and used SB live 24bit with device setting=24b 48K, recorded ion goldwave in 24bit 48K mode)of clavichord music, which is not using dull dynamic range(clavichord is a quiet instrument)

My question is that how can I down sample it to 16 bit with full 16bit dynamic range, I mean is the wave peak in 24bit file(e.g. assign 8664687/16771216 to be 65536/65536)?

I don't think it's maximizing

Is it one of the XRCD principle?

If I undestand your post you essentially want to adjust the gain of the file so that it peaks at digital fullscale.

Rather than dithering a 24-bit WAV to 16-bits, wouldn't it be better to encode it to FLAC (or some other lossless codec), calculate the peak, and set the ReplayGain appropriately? You could still realize the same loudness benefits without the degradation in quality from dithering.

If you scale the audio to floating point and calculate the peak sample as x, you would want to set the ReplayGain to 20 * log(1/x) dB. For the example you give above, the RG would be +5.74 dB (5.73623346623 by my math).

Since i am going to produce a 16 bit file, i would like to dither it and assign the 8664687/16777216 to plain 65536/65536 instead of replaygain

step 1: normalize the file. If you want the maximum value to be used, normalize to 0dB.
step 2: resample to 16 bit, using noise-shaped dither.