IOP data


1. Data processing notes

The Inherent Optical Properties (“IOP”) data are acquired with a so-called IOP frame, which is a stainless steel frame onto which several IOP sensors are fixed, and connected to a single data logger. The frame is then lowered at sea or trough a hole in the ice floe, with the help of a motorized pulley, and data are recorded on the way down (optimum configuration) to 200 m approximately, as well as on the way back to the surface (sub-optium configuration). The suite of sensors that were installed for the GreenEdge 2015 and 2016 field campaigns are the following:

  1. An AC-S, from WetLabs, acquiring attenuation (“c”) and absorption (“a”) hyperspectral spectra,
  2. A Bb9, from WetLabs, acquiring back scattering coefficients at 412, 440, 488, 510 532, 595, 650, 676 and 715 nm,
  3. A Bb3, from WetLabs, acquiring back scattering coefficients at 720, 770 and 870 nm,
  4. A FLBBCD, From WetLabs, acquiring back scattering coefficient at 700 nm, as well as fluorescence of Chl-a and fluorescence of CDOM,
  5. A FastCat 49 pumped CTD, from SeaBird Electronics, acquiring temperature and salinity,
  6. A LISST-Deep, from Sequoia Scientific, measuring particle size distribution into 32 size bins.

Note: LISST-Deep data have not been processed yet, and should be delivered in the next release of this IOP data base.

    1.1. WETLabs factory softwares

        1.1.1. WAP (parse raw binary files)

The raw files, which are binary ones, have first to be parsed with a set of WETLabs specific softwares. The First, called “WAP” (for WETLabs Archives Processing), converts binary files into either ASCII or others binary files, and apply the factory calibration coefficients.

        1.1.2. COMPASS (merges CTD and AC-S data)

The second WETLabs software, called COMPASS, is intended to convert the AC-S binary file into an ASCII one, apply sea water temperature and salinity corrections to the absorption and total attenuation coefficients, and also to merge the AC-S and the CTD files, based on timestamps (the timestamps are imposed by the data logger at the time of the data acquisition).

ACS-CTD merged files will act as the reference files onto which all the others sensors data will be merged, via a set of CRAN R routines.

COMPASS also provides an option to apply a pure water offset correction, but as it is not very flexible, this correction is performed within the custom R codes. COMPASS also provides the option to apply the residual scattering error correction, but as this has to be applied after any pure water offset correction, this option is not used either under COMPASS (see algorithm section below), but agin, this is performed within the custom R routines. COMPASS is then used only to merge the CTD and the AC-S data, as well as to apply the sea water temperature and salinity corrections.

    1.2. In-situ AC-S pure water calibration

Regular in-situ pure water calibrations are very important for the AC-S sensor, as it known to be a drifting instrument. During the GreenEdge ice camp sampling efforts, pure water calibration was performed on each sampling day, and a linear regression as a function of time was computed for each wavelength of absorption “a” and attenuation “c” signals. Then, the offset applied during the data processing was taken on this linear regression at the exact date of the measurement. See the figure below for illustration.

Spectra of all the performed calibrations (top: a - bottom: c, note the extreme sensibility of the offset to the temperature in the longer wavelengths, typically above 700 nm):

Linear regression performed on “a” offsets at selected wavelengths:

Linear regression performed on “c” offsets at selected wavelengths:


    1.3. Main R routines

The following algorithm is used to process each set of input files:

  1. Reads all the input files,
  2. Merged all the sensors data to the ACS/CTD ones, based on time stamps. The time window used is +/- 1 second. All the data found +/- 1 second of the ACS-CTD data are averaged, using the “mean” function,
  3. Compute and apply ACS pure water offsets (with the linear fit explained above),
  4. Compute a moving average of the ACS data for the next step (only), as recommanded by WETLabs,
  5. Compute the residual scattering error. If possible, use the Zaneveld method (see method #3 of the ACS user’s manual). In some cases, especially where the water is very clear, the correction diverges (“zero / zero”), and whenever the correciton is larger than the value to correct, this method is flagged as unapropriate and a blank correction is applied, removing the siganl at 680 mn. The wavelength of 680 nm has been chosen because its sensitivity to temperature effect is less important than higher wavelengths (715 for example),
  6. Compute the bbp values from the measured Beta values, using the code below,

The code used to compute the bbp values from the Beta values is the following:

# Salinity and absorption correction for BB data
    #-----------------------------
    cat("salinity and absorption correction to Beta values:\n")
    # 
    deltas <- array(NA, dim = c(nline.acs,13))  
    for (cpt in 1:13) {
      deltas[1:nline.acs,cpt] <- 1.38 *
        (waves.all.bb[cpt] / 500)^(-4.32) *
        (1 + 0.3 * sal[1:nline.acs] / 37)*(1e-4) *
        (1+ (cos(117 * pi / 180))^2 * (1-0.09)/(1+0.09))
    }

    '- "nline.acs" is the number of lines of the ACS file,
     - "13" is the number of bbp channels,
     - waves.all.bb is an array containing all the bbp channels wavelengths,
     - sal[1:nline.acs] is an array containing all the salinity values form the CTD, merged on the ACS data.
    '
    
    cat("720 nm, ")
    bb3.res[,1] <- (bb3.res[,1]  * exp(0.0391*d.acs.w.avg.resscat[,idx.a.720]) * 2 * pi * 1.1)
    bb3.res[,1] <- bb3.res[,1] - deltas[,11]
    
    '- "bb3.res[,1]" contains the bb3 data at channel 1 (720 nm),
     - "d.acs.w.avg.resscat[,idx.a.720]" contains the ACS data at channel 720, after the pure water offset has been applied, as well as the moving avergae and the residual scattering error correction.
    '
   
    # and so on for the other wavelenghts

    1.4. Cleaning and homogemeization of the output data

AC-S wavelengths are slightly modified from one factory calibration to the next one. For example, the first attenuation wavelengths logged in the March 2015 factory calibration file are given as 400.9 nm, 405.0 nm, 409.3 nm, 413.2 nm, … whereas in the December 2015 factory calibration file, the attenuation wavelengths are given as 401.2 nm, 404.3 nm, 408.9 nm, 412.6 nm, …

In order to get an homogeneous output from one year to the other, the a and c spectra are then interpolated and given on a 5 nm step grid. See the figure below that shows typical example of a and c spectra.

Interpolation (every 5 nm) of the “a” signal:

Interpolation (every 5 nm) of the “c” signal:

Also, for all the sensors, a clean-up of the data is performed to remove spkikes and outliers (which may occur when a large particle moves in front of a sensor field of view). To do so, a running median is computed at each measurement point, taking 25 measurements before and 25 measurements after this point. A local regression algorithm (“loess”) is then performed as a function of the depth, not only to smooth a little bit further the profiles, but also and especially to give the output data on a 0.25 m depth grid step. The following figures show examples of this data cleaning protocol. See each figure’s legend to find the raw data, the running median data as well as the smoothed data.

Cleaning of the “a” signal

Cleaning of the “c” signal:

Cleaning of the CDOM fluorescence signal:

Cleaning of the Chl-a fluorescence signal:

    1.5. Averaging profiles

For each station, which is equivalent to each sampling day for both the 2015 and 2016 ice camp sub-cruises, an averaged profile has been computed. Note that an average has been computed for both the downcasts and the upcasts, separately. Indeed (see the metadata sheets which give more details), the IOP frame user’s guide indicates that data acquisition should be performed on the downcast only, as some of the sensors might be afftected by the water drag created in the frame’s wake on the way up. Upcasts have nevertheless been recorded as few sensors are not affected, namely the ACS and the CTD.

The figure below (produced with a code from Philippe Massicotte) show, for illustrative purposes, the averaged profiles of the absorption coefficients (“a”), the attenuation coefficient (“c”) and the particles back scattering coefficients (“bbp”), for both the 2015 and the 2016 ice camps data, for the downcasts only.

Ice Camp 2015 - averaged “a” profiles:

Ice Camp 2015 - averaged “c” profiles:

Ice Camp 2015 - averaged “bbp” profiles:

Ice Camp 2016 - averaged “a” profiles:

Ice Camp 2016 - averaged “c” profiles:

Ice Camp 2016 - averaged “bbp” profiles:

2. Directories and files content

    2.1. Directories content

For both the 2015 and 2016 ice camp sub-cruises, the data are oganized in the same way (the Takuvik IOP frame has not been deployed on the Amundsen sub-cruises, see the UQAR IOP data). Namely, in each of the ./GE2015.ICMP/ and ./GE2016.ICMP/ sub-directories:

  • the LOG/ directory provides 2 log files, a more detailed one (LOG.GE2016.ICMP.IOP.long.xlsx), giving the calibration files acquisition details, and a shorter one (LOG.GE2016.ICMP.IOP.short.csv), giving only the details of the main profiles files,

  • the RES.ALL/ directory provides data for all the individual profiles. The file naming convention is the following: GE201X.ICMP.201Xddd.tyy_IOP.csv where X stands for the year (5 for 2015 and 6 for 2016), ddd stands for the day of year, t stands for the cast type (d for downcast and u for upcast), and yy stands for the cast number. For example, the cast called GE2015.ICMP.2015104.d01_IOP.csv is the first downcast of 2015 ice camp data, from day of year 104,

  • the RES.AVG directory provides data of averaged profiles. An averaged profile has been computed for each station, which is for each day during the ice camps sampling activities. The file naming convention is slightly different but contains the same information, for example the GE2015.ICMP.2015104.d.csv file contains the averaged profile data of all the downcasts of the day 104 of the year 2015. Note that the averages have been computed for downcasts and upcasts separately, as some of the sensors are meant to be acquiring data on the downcast only (as per their user’s guide). See the metadata sheets which give more details. Also note that the averaged downcast and the averaged upcasts are given in the 2 sub-directories called DOWNCAST and UPCAST respectively.

    2.2. Files content

The log files parameters are rahter self-explained. Here we provide especially details on the content of the data files.

        2.2.1. Individual profiles data files

The files from the RES.ALL/ folder provide the following parameters:

  • date-UTC    : UTC date with the YY-MM-DD format,
  • time-UTC    : UTC time with the HH:MM format,
  • doy              : day of the year, ranging from 1 to 366,
  • filename    : the filename that was output by the main data processing code,
  • lat              : approximate latitude (ice camp polarhaven one),
  • lon              : approximate longitude (ice camp polarhaven one),
  • cast_type  : type of the cast, namely either downcast or upcast,
  • depth.grid: depth, from the 0.25 m stepped grid, at which the data is provided (smoothed),
  • a_lll          : absorption coefficient at wavelength lll nm, given in m-1,
  • c_lll          : attenuation coefficient at wavelength lll nm, given in m-1,
  • bbp_lll      : particles back scattering coefficient at wavelength lll nm, given in m-1,
  • temp_c        : temperature, given in Celsius,
  • sal_psu      : salinity, given in p.s.u.,
  • fluo_chla  : Chlorophyll-a concentration, given in mg.m-3,
  • fluo_cdom  : CDOM concentration, given in p.p.b.

        2.2.2. Averaged profiles data files

The files from the RES.AVG/ folder provide the following parameters:

  • date                    : UTC date with the YY-MM-DD format,
  • doy                      : day of the year, ranging from 1 to 366,
  • lat                      : approximate latitude (ice camp polarhaven one),
  • lon                      : approximate longitude (ice camp polarhaven one),
  • cruise                : sub-cruise from which the data are coming from,
  • cast_type          : type of the cast, namely either downcast or upcast,
  • depth.grid        : depth, from the 0.25 m stepped grid, at which the data is provided (smoothed),
  • mean_a_lll        : mean absorption coefficient at wavelength lll nm, given in m-1,
  • mean_c_lll        : mean attenuation coefficient at wavelength lll nm, given in m-1,
  • mean_bbp_lll    : mean particles back scattering coefficient at wavelength lll nm, given in m-1,
  • mean_temp_c      : mean temperature, given in Celsius,
  • mean_sal_psu    : mean salinity, given in p.s.u.,
  • mean_fluo_chla: mean Chlorophyll-a concentration, given in mg.m-3,
  • mean_fluo_cdom: mean CDOM concentration, given in p.p.b.