Definition of the Depth Bin Index and Offsets
Data Processing – Acoustic Back-Scatter (ABS)
Data Processing – Velocity and Shear
Iridium SBD Binary Data Format
The SBD TEXT Format for ADP data (XXXX.TXT)
Conversion from ‘VHEX.TXT’ to ‘adpYYMXXXNN.ADR’
The individual scan line format:
Sontek ADP Beam Orientation in the Spray:
Version Updates:
09Mar07 merged all ADP documentation (binary/text/flash) into this document.
15Mar07 Updated ‘Vhex.txt’ and ‘.ADR’ file description for the newer versions.
23Apr07 0702b modified adp.c to process at 0.1 mm/s resolution (old=1 mm/s),
and then pack at 1 mm/s : NO difference at the shore-processing end.
24Apr07 .ADR file now has bottom depth estimates from the altimeter readings.
The Spray uses a Sontek Argonaut 750 kHz three-beam system in a profiler (multi-range-cell) mode, which is referenced here as the ADP ( Acoustic Doppler Profiler). The ADP is mounted in the tail such that a 17 degree ascent pitch angle with zero roll results in the ADP central axis pointing straight down.
The parameters that define the ADP operation are
(The following are set in the code at the time of compilation and are in the CD58 cal. line):
BD Blanking distance, is hardwired in the code to 2.0 m.
K1_ABS, K2_ABS Start/End cell indices to use for the Acoustic Back-Scatter (ABS).
Since 3/2006, code is hardwired to K1_ABS=2, K2_ABS=3.
These values are also stored in the calibration header lines.
(The following are stored in EEPROM, can be changed from shore and are in the B1 line):
CELL_SIZE Sample cell size [m], as measured along the central axis.
PULSE_LENGTH Pulse length [m], as measured along the central axis of the ADP.
NCELL Number of cells sampled in range for each ensemble.
ZMAX Max depth [m] to start acquiring ADP data on ascent.
The nominal settings are 4 m cell size, 8 m pulse length, 5 cells, with ZMAX = 500 m.
A 16-ping ensemble average is performed every CELL_SIZE meters on ascent. Let each ensemble-average be denoted by an iscan index number. ADP ensemble indices are:
iscan Ensemble-average scan index ( = 1.. nscan).
nscan Total number of ADP scans in the ascent profile.
ibeam Beam index (for beam 1, 2, or 3).
icell Cell index (=1 ..NCELL). This is synonymous with ‘range bin.’
For each iscan, the following ADP data are recorded (and stored to FLASH card as well):
Vel[ibeam][icell] Along-beam velocity [ mm/s] for each beam and each cell.
Noise[ ibeam ] Noise floor [counts] for each beam.
Amp[ibeam][icell] Return amplitude [counts] for each beam and cell.
The Spray’s attitude measurements are also recorded for each iscan:
ADP_P [iscan] Spray’s pressure at the time of the ADP iscan.
ADP_Pitch [iscan] Pitch angle (positive is nose up).
ADP_Roll [iscan] Roll angle (positive is port wing up).
ADP_Head [iscan] Compass Magnetic Heading.
As well, the 3-beam average of the last cell’s return amplitude is also computed:
ADP_A [ iscan ] Average amplitude across all three beams for the last cell.
This is a good measure of signal strength.
At the end of the profile, on-board processing produces depth-profile estimates of:
U [ kz ] East velocity [mm/s].
V [ kz ] North velocity [mm/s].
W [ kz ] Vertical velocity [mm/s].
ABS [ kz ] Acoustic Back-Scatter [ dB ], using only cells K1_ABS to K2_ABS.
The most-shallow ADP ensemble is from the last scan. Let
P0_Spray = ADP_P[ nscan ] . = most-shallow depth.
The center of the first cell is
offset downwards from the Spray, dependent upon the blanking distance, pulse
length and cell size, given by DP1:
DP1 = BD + (CELL_SIZE + PULSE_LENGTH)/2 .
From which the center of any cell is given by
DP[ icell ] = DP1 + CELL_SIZE*( icell – 1) .
To depth-average the data across the different scans, cells that overlap in the same depth bin have their values accumulated, and then averaged at the end of processing all scans. The depth bin index for the first cell for each scan is given by:
kz
[iscan] = round( ( ADP_P[
iscan ] – P0_Spray )/CELL_SIZE); =First
cell index.
Note, this means if Spray moves slowly between scans, they will be averaged into the same depth bin.
The Amp[ibeam][icell] is
averaged across beams and corrected for
spherical spreading and sound absorption as a function of range to give an
estimate of ADP_ABS:
ADP_ABS[icell] = Beam_average(Amp[icell] ) + 20*log10( DP[icell] ) + ALPHA*2*DP[icell].
where ALPHA = 0.22 (hardwired in the code) is the sound absorption, and
DP[icell] = DP1 + (icell-1)*CELL_SIZE is the depth below Spray of icell .
ADP_ABS is computed for the subrange of icell = K1_ABS to K2_ABS.
The ADP_ABS values are then accumulated into the ABS array:
ABS[ kz + icell-1 ] = ABS[kz+icel-1l] + ADP_ABS[ icell ] for icell = K1_ABS to K2_ABS.
N_ABS[ kz + icell-1 ] = bin counter, is incremented as well.
At the end of the processing, the average is taken ( ABS[kz] = ABS[kz ] / N_ABS[kz] ).
NOTE: For K1_ABS=2, the most-shallow bin has no data.
For K1_ABS=3, and ncell=5, the last two depth bins of ABS have no data.
Starting with the top scan, the
along-beam velocities are used to compute [U,V, W] in the Earth-Coordinate
system (using the Spray Pitch, Roll and Heading).
The horizontal velocities U and V are first-differenced in depth for the shear estimate:
DU[iscan][ icell ] = U[iscan][ icell+1] – U[iscan][icell]. (done for both U and V).
The shears are then added to accumulators, using kz[iscan ] to sort into the right depth bin.
ADU[kz+icell-1] = ADU[kz+icell-1] + DU[icell] (icell = 1..ncell-1 ).
The average is taken after all scans are processed (for both ADU and ADV ).
If each scan is correctly sampled every CELL_SIZE m, then there should be ( ncell-1 + nscan-1 ) shear estimates. Since the deepest scan is the first, and requires some initialization time, it is actually taken at a shallower depth than desired, and the next ADP scan occurs sooner to compensate. Therefore, many times the two deepest scans actually get sorted into the same depth bin kz, and the total number of bins reported is one less than expected (this problem is corrected in code 0702).
To allow the shear profile ADU to use the same packing algorithm as used for every other sensor (P, T, S,OPT, ABS, etc.), it is integrated before transmitting.
Let AUU[0] = 0, (set the top bin velocity to 0) and then integrate:
AUU[ kz ] = ADU[ kz ] + AUU [ kz –1 ], for kz=1 to last valid kz.
giving a velocity profile with
an arbitrary offset so the top value is always 0. Since the top value is always 0, the Spray code actually does not
transmit this value, but as of Feb07, the shore-processing has changed to
insert a zero value for the first AUU, AVV.
Therefore, the first reported velocity (=0) in the text file is of the
most shallow depth bin ( PO_Spray + DP1 ).
Each SBD message contains multiple sensor data packets, with each packet
having a unique sensor ID byte.
For the ADP, the sensor ID byte is 5x, where x=0-F, and the
following packets are defined:
ID Parameter Description
50 ADP_P Pressure
of Spray at each scan ensemble.
51 AUU East velocity [ mm/s] , >0 for east flow.
52 AVV North
velocity [mm/s], > for north flow.
53 AWW Vertical velocity [mm/s], > 0 for upwards flow
(presently not sent back).
54 ADP_A Avg. amp of the last bin of each scan [counts; 1 count =
0.43 dB].
55 ADP_Pitch Pitch at each scan [counts: 1 count = 0.4 degrees].
56 ADP_Roll Roll at each scan [ counts: 1 count = 0.4 degrees].
57 ADP_Head Compass Heading, magnetic [ counts: 1 count = 0.1
degrees].
58 ABS Acoustic Back-Scatter [ counts: 1 count = 0.1 dB ].
5F Settings The
present ADP settings (see ‘B’ line below).
All ID except 5F are sent in the standard data-packing algorithm (time-series first-differenced and then packed with an auto-scalar for each 20-bin segment; details are provided in the SBD_Binary document). Packet 5F contains 4 bytes with the following values:
byte 1 = NCELL = number of cells sampled in range for each ensemble.
byte 2 = CELL_SIZE= sample cell size [m].
byte 3 = PULSE_LENGTH = pulse length [m].
byte 4 = MIN_SNR = Minimum Signal-to-Noise-Ratio [counts = 0.43 dB/count] to use.
Cells with SNR values below this value are not used in the calculations of velocity or backscatter. If MIN_SNR = 255, ALL cells are used for shear, no matter the SNR values. This packet is parsed into the ‘B1’ line in the XXXX.txt file.
The binary data is immediately parsed by the ‘hex2tmp’ program, which writes out the new information to the XXXX.txt file.
As with the rest of the SBD binary data, the bytes are unpacked and stored in ascii format in the XXXX.txt file, where XXXX=Spray serial number. Documentation for the general .TXT file can be found in the ‘Spray_TXT_0702’ file. The following lines are a direct cut-and-paste from that file:
‘B1’
line Contains the following Doppler parameters
B1 17 5 4 8 255 8.0 0 1 4
12345678901234567890123456789012
1 2 3
B1 17 5 4 8 255 8.0 0 1 4
i=
1 2 3 4 5 6
7 8 9
i1 =dive number = s[4:7].
i2 =NCELL = number of cells (bins) for
each ensemble-average = s[9].
i3 =CELL_SIZE = cell size [m], as
measured along the central axis = s[11].
i4 =PULSE_LENGTH = pulse length [m],
as measured along the central axis =s[13].
i5 =MIN_SNR = Minimum
Signal-to-Noise-Ratio [counts] to use to decide if the data is ‘good’. One
count = 0.43 dB. If = 255, then MIN_SNR
is NOT used. =s[15:17].
New 1/26/07
i6 = DP1 [m] : Depth offset from Spray to the
center of the first bin = s[20:23].
i7 = V_off : #bins offset from DP1 to the
first returned velocity value =s[25:26].
i8 = ABS_off : #bins offset from DP1 to the
first returned ABS value = s[28:29].
i9 = AMP_off : #bins offset from DP1 to the
first returned ADP_A value = s[31:32].
‘AT’ Doppler profiler data
follows: The ‘AT’ line gives the type
of data and number of points.
AT07 289 15 12 0
xx i1 i2 i3 i4
xx =
Sensor Type:
Here is the quick summary (see the
Spray_ADP.doc for the complete description):
0 ADP_P Pressure of Spray at each
scan ensemble.
1 AUU East velocity [ mm/s] ,
>0 for east flow. First bin is set=0.
2 AVV North velocity
[mm/s], > for north flow. First bin
is set=0.
3 AWW Vertical velocity [mm/s],
> 0 for upwards flow.
4 ADP_A Avg. amp of the last bin of
each scan [counts; 1 count = 0.43 dB].
5 ADP_Pitch Pitch at each scan [counts: 1
count = 0.4 degrees].
6 ADP_Roll Roll at each scan [ counts: 1
count = 0.4 degrees].
7 ADP_Head Compass Heading, magnetic [
counts: 1 count = 0.1 degrees].
8 ABS Acoustic Back-Scatter [
counts: 1 count = 0.1 dB ].
i1 = dive number;
i2 = number of datum that follow in the ‘a’
lines.
i3 = number of columns per ‘a’ line.
i4 = SBD packet ID (=0 if sent in the first
packet, 1 if sent in a second packet).
The ‘AT’ line is immediately
followed by NLINES = ceiling (i2 / i3) lines of the profile data, with each
line beginning with an ‘a’.
a 523
558 590 607
587 567 537
493 470 412
377 346
a 352
402 428
Interpreting the ADP .TXT
arrays.
The depth of the first bin and the expected number of points for each sensor type is given below:
ID Parameter #pts Depth
of First bin
0 ADP_P nscan P0_Spray (= Spray depth of the shallowest ADP scan).
5 ADP_Pitch nscan P0_Spray.
6 ADP_Roll nscan P0_Spray.
7 ADP_Head nscan P0_Spray.
AMP_off = NCELL – 1 (also given in the ‘B1’ line above).
4 ADP_A nscan P0_Spray + DP1 + AMP_off * CELL_SIZE.
ABS_off = K1_ABS –1 (also in the ‘B1’ line).
n_ABS
= K2_ABS + nscan – K1_ABS (max value) .
8 ABS n_ABS P0_Spray + DP1 + ABS_off * CELL_SIZE.
(data pre-0702 have been post-processed to match the above).
n_U
= NCELL + nscan –1 (max value).
V_off
= 0 ( also in the ‘B1’ line).
1,2 AUU, AVV n_U P0_Spray
+ DP1 + V_off * CELL_SIZE.
(above is for the newer code, where the first value=0).
If some scans overlap in depth bins, then both n_U and n_ABS will be less than the maximum values given above. Typically, the deepest two scans overlap, and the rest are spaced correctly, so n_U and n_ABS are one less than the maximum value listed above.
Each 16-ping ensemble average of the Sontek ADP data is written to the flash card, along with pressure, heading, pitch, and roll. For every surfacing a new file is created on the flash card. At the end of the mission these binary files are converted to one large hex file, ‘VHEX.TXT’, (program ‘Spray_v2hex’) where each file’s data set is delimited by the ‘V’ line:
V 0001 5
4 8 8.0 0001 251 2006 201 1384
4 17.9717 +36.79334 -121.83425
i= 1
2 3 4 5 6
7 8 9 10 11
12 13 14
i1 = Vxxx file number (can
contain multi-dive profiles).
i2 = NCELL.
i3 = CELL_SIZE [m].
i4 = PULSE_LENGTH [m].
i5 = DP1 [m] = center
depth offset of the first cell.
i6 = Spray Dive number.
i7 = Pressure Surface
counts (from the .sat file).
i8 = Year
i9 = Julian Day (01Jan =
1).
i10= GPS week number.
i11= GPS week day.
i12= GPS HH.HHH UTC (decimal
hr).
i13= GPS Lat.
i14= GPS lon.
This is followed by NSCAN lines of the scan data for that file, in hexadecimal format:
0a3c003a2705d7ffd8ffe0ffeeffe7ffbf00d0...000675745382e6f5c4c42366c604e453f
Let
A = character string of the hexadecimal data.
N = number of characters in the array A.
NCELL = number of cells for the Doppler data (gotten from the V line, above).
Nbeams = 3 = number of beams.
Nxxx = # of hex characters used to store one datum, dependent on data type:
NAMP= 2 characters for amplitude of the return signal.
NSTD= 2 characters for standard deviation (presently not used; values set to 0).
NVEL= 4 characters for along-beam velocity.
NN = NCELL * Nbeams * Nxxx. = # of characters to store (xxx=AMP,STD,or VEL).
N = 12 + (NCELL*Nbeams)*( NAMP + NSTD + NVEL) = total #char in A.
The first 12 characters are the header information:
A( 1:4 ) = ADP_P (counts-to-dBar is the same as with the satellite data).
A( 5:8 ) = ADP_Head = Heading counts : Degrees = counts/10.
A( 9:10)= ADP_Pitch = Pitch counts.
A(11:12)= ADP_Roll = Roll counts. Both pitch and roll convert to degrees in the same way:
if ADP_Roll>128, ADP_Roll = ADP_Roll – 256. This converts it to a signed value.
Degrees = ADP_Roll *0.4
A(13:N) contain the velocity and intensity data for all 3 beams, and all NCELL.
A(13:NV) = Velocity data; NV = NCELL*Nbeams*NVEL + 12.
A(NV+1:NS) Std. Dev data (not valid): NS = NV + NCELL*Nbeams*NSTD.
A(NS+1:N) = Amplitude data.
The hex data above is unpacked into an ascii format, easily readable for the next stage (c program ‘vhex2adr’). This output file name is:
adpYYMXXXNN.ADR
where YYMXXXnn is the same format as the ‘.sat’ file name.
YY = Year
M = Month (1..C. A=Oct, B=Nov, C=Dec).
XXX = Spray s/n
NN = mission i.d. (typically = 01 ).
The start of each dive is
delimited by the line (updated Apr07 with altimeter
info)
-99 41
268.45911 15 5
2.0 4.0 8.0
8.00 12.00 16.00 20.00 24.00
72 3 2 0 0...
i=1 2 3
4 5 6 7 8
9 10 11
12 13 14
15 16 ...
where for index i:
i1 = -99 = delimiter flag that this is the start of a new dive.
i2 = idive = dive number.
i3 = Julian Year-Day, from the GPS start-of-dive fix.
i3 = -99.0 means this is a dive in a multi-dive sequence between GPS fixes.
i4 = nscan = # of 16-ping averaged ensembles collected for this dive.
i5 = NCELL = # of range cells (range bins) in each ensemble.
i6 = BD = blanking distance (meters, measured along the central axis of the Doppler).
i7 = CELL_SIZE (meters, measured along the central axis) used to sample the Doppler shift.
i8 = PULSE_LENGTH ( meters, measured along the central axis).
i9+j = cell_offset(j) = the offset (meters, measured along the central axis) between the Doppler transducers and cell #j. The center of the first cell is located at:
DP1 = BD + (CELL_SIZE + PULSE_LENGTH )/2.
Each successive cell is offset by CELL_SIZE.
(added apr07 : parameters
regarding guessed bottom depth from the altimeter).
i9+NCELL = ALT_BOTTOM [m] = guess of bottom depth from the altimeter readings.
i9+NCELL+1 = QF = Quality flag
0=no altimeter data.
1=Spray was sitting on the bottom (so we know bottom depth quite well!).
(either didn’t turn around in time and/or altimeter mode was not on).
2+STD = 2 + Standard deviation [m] of the last altimeter reading compared
the rest taken while approaching the bottom.
i9+NCELL+2 = N_ALT = number of altimeter samples used for the estimate of STD .
The rest of the ‘-99’ line is zero-filled, so all lines have the same number of columns. This line is then followed by nscan lines of the data, followed by the next ‘-99’ for the next dive, etc.
4.2
15.6 0.0 278.0 45
45 43 87 117 100 112 104
93 92 114 96
-23 ...
i= 1
2 3 4 5 6
7 8 11 14 17 ...
where
i1 = P = Pressure [dBar] of the Spray at the time of the ensemble-average.
The first cell’s center depth is equal to: P + cell_offset(1).
i2 = Pitch [degrees], >0 is nose-up.
i3 = Roll [degrees], >0 is port-wing up.
i4 = Heading [degrees] (magnetic).
i5,6,7 = Return Signal(amplitude) noise level [counts] for beam 1, 2, and 3.
If 0, then no noise level was recorded (first used in Feb 2006).
Amp[ibeam, icell] data follows for the next 3*NCELL columns (=return amplitude [counts]).
i=8 + (j-1)* NCELL = Amp [j, 1] = First cell location for beam # j ( j=1, 2, 3).
i=8 + (j-1)* NCELL + (k-1) = Amp [j, k] (counts) for beam j, cell k (k=1.. NCELL).
To convert from counts to decibels:
[ decibels] = Amp *0.43.
Vel[ibeam, icell] = Along-Beam Velocity data are in the next 3* NCELL columns.
Velocity > 0 is AWAY from the beam (this is the Sontek standard sense).
i=8 + NCELL *3 + (j-1)* NCELL for j=1..3 is the first cell location of velocity for each beam=j.
i=8 + NCELL *3 + (j-1)* NCELL + (k-1) = Vel [ j, k ] [mm/s] for beam j, cell k(k=1.. NCELL )
The Sontek Doppler’s Return amplitude is from a log-amplifier. Therefore the 16-ping ensemble-average is an average of the log values. This will cause an inherent low bias in the estimate of the average backscatter strength, where the amount of bias is dependent upon the backscatter’s pdf.
In the above, distances are measured along the central axis of the sonar, not the slant range. Therefore spherical spreading plus sound absorption losses should be adjusted accordingly (beam angle with-respect-to the central axis = 25 degrees).
Beam 1 = forward.
Beam 2 = port aft.
Beam 3 = starboard aft.
The ADP central axis is oriented such that with a Spray pitch of 17 degrees nose up, 0 roll, the central axis is straight down. The beams are 120 degrees apart azimuthally, and 25 degrees off of the central axis; at Spray pitch = 17 degrees nose up, the forward beam (beam 1) is in line fore-aft with the Spray, and is pointing 25 degrees forward (with respect to the vertical).
Let CT, ST = cosine and sine of the transducer angle = 25 degrees (with-respect-to vertical).
vb1, vb2, vb3 = along-beam velocities for the 3 beams (NOTE: v>0 for velocities moving AWAY from the transducer).
vg1, vg2, vg3 = along-beam velocities transformed to the vehicle coordinates:
vg1 = long-axis (fore-aft) velocity (positive flow from tail-to-nose).
vg2 = cross-axis, in the wing plane (positive flow from stbd to the port wing).
vg3 = cross-axis, in the tail plane (positive flow upwards).
(For a normal ascent, vg1 ~ -30 cm/s, vg2 ~ 0, vg3 ~ 10-15 cm/s ).
vg1 = ( 2*vb1 - vb2 - vb3 ) / ( 3* ST);
vg2 = ( vb2 - vb3 ) / ( sqrt(3) * ST );
vg3 = -( vb1 + vb2 + vb3 ) / (3 * CT);
Remove the roll.
CR, SR = cosine, sine of roll , roll is >0 for the starboard wing down.
vr1 = vg1 = long-axis vehicle velocity: no change for roll correction.
vr2 = cross-axis (wing plane) corrected for roll (now in the horizontal plane).
vr3 = cross-axis (tail plane) corrected for roll.
vr2 = vg2 * CR – vg3 * SR;
vr3 = vg2 * SR + vg3 * CR;
Remove the pitch
CP, SP = cosine, sine of pitch , pitch is >0 for the nose up.
vp1 = long-axis velocity, now in the horizontal plane.
vp2 = vr2 (not affected by pitch).
vp3 = tail-plane corrected for pitch (now vertical velocity).
vp1 = vr1 * CP – vr3 * SP;
vp3 = vr1 * SP + vr3 * CP;
Correct for heading:
CH, SH = cosine, sine of heading: 0 = North, 90 = East.
vN = north velocity.
vE = east velocity
vW = vertical velocity = vp3.
vE = vp1 * CH – vp2 * SH;
vN = vp1 * SH + vp2 * CH;
Comparison of flash card to satellite data showed some discrepancy due to the Spray code computing shear at 1 mm/s resolution and integrating to velocity, thus lacking better resolution for the integration.
This problem was handled by increasing the resolution to 0.1 mm/s at the time that the along-beam velocities are transformed to vehicle coordinates. The starting along-beam resolution from the ADP is 1 mm/s, and accuracy of ~3.3 mm/s along-beam, 5.5 mm/s for the horizontal components. They are kept at 0.1 mm/s resolution for the remainder of the processing, up through integration to give a velocity profile. At that time the resolution is lowered back to 1 mm/s for the satellite data transmission. This makes data interpretation backwards-compatible with previous deployments.
Search adp.c for ‘0702b’ to see the code changes.
Potential problems: the temporary velocity array between transformations is a 2-D ‘short’. The 0.1 mm/s resolution gives bounds of +/-32768 = 327 cm/s range bounds. Given the glider’s ~25 cm/s relative speed through the water, this should never be a problem.
First glider to be deployed with this updated code is Spray 13 (Line 90) on 25Apr07.