I07 reflectometry data

Detailed here are the techniques one can use to reduce reflectometry data acquired on the I07 beamline at diamond using the islatu package. Both the vertical scattering and the double-crystal deflector geometries are discussed. The particular example of reduction of reflectometry data collected from a water-air interface is worked through in detail.

At I07, a reflectometry profile is generally constructed from multiple scans. Each scan has a corresponding nexus file (.nxs) which contains metadata, and an hdf5 file (.h5) containing the images recorded in the scan.

The first task is to instantiate an instance of islatu’s Profile class containing the reflectivity profile’s data. To do this, we need to download the profile’s constituent .h5 and .nxs files, and tell islatu where to find the .nxs files. Provided that your .h5 files are in the same directory as your .nxs files, islatu will be able to load your data.

from islatu.refl_profile import Profile
from islatu.io import i07_nxs_parser

# On my machine, I have some .nxs and .h5 files in this directory.
data_dir = "../../tests/resources/"
# I have scans 404875-404882 stored in the above directory.
scan_numbers = range(404875, 404882 + 1)

data_files = [f"{data_dir}i07-{i}.nxs" for i in scan_numbers]

# Profile.fromfilenames() expects a list of paths to data files, and a parser
# from islatu.io that it can use to parse the data files.
my_profile = Profile.fromfilenames(data_files, i07_nxs_parser)

Data file found at ../../tests/resources/excaliburScan404875_000001.h5.
Loading images from file ../../tests/resources/excaliburScan404875_000001.h5
Currently loaded 51 images.
Data file found at ../../tests/resources/excaliburScan404876_000001.h5.
Loading images from file ../../tests/resources/excaliburScan404876_000001.h5
Currently loaded 8 images.
Data file found at ../../tests/resources/excaliburScan404877_000001.h5.
Loading images from file ../../tests/resources/excaliburScan404877_000001.h5
Currently loaded 8 images.
Data file found at ../../tests/resources/excaliburScan404878_000001.h5.
Loading images from file ../../tests/resources/excaliburScan404878_000001.h5
Currently loaded 9 images.
Data file found at ../../tests/resources/excaliburScan404879_000001.h5.
Loading images from file ../../tests/resources/excaliburScan404879_000001.h5
Currently loaded 13 images.
Data file found at ../../tests/resources/excaliburScan404880_000001.h5.
Loading images from file ../../tests/resources/excaliburScan404880_000001.h5
Currently loaded 14 images.
Data file found at ../../tests/resources/excaliburScan404881_000001.h5.
Loading images from file ../../tests/resources/excaliburScan404881_000001.h5
Currently loaded 31 images.
Data file found at ../../tests/resources/excaliburScan404882_000001.h5.
Loading images from file ../../tests/resources/excaliburScan404882_000001.h5
Currently loaded 26 images.

When we used the fromfilenames class method above, islatu loaded all of the images taken in the scan. It’s now possible to use the profile to get a very crude indication of what our data looks like. This is possible because islatu uses the default signal and default axis specified in the nexus file to guess a profile’s reflectivity as a function of Q or θ.

# Note that any instantiated islatu profile has the following properties:
print(f"Theta values: {my_profile.theta}")
print(f"Corresponding q-vectors: {my_profile.q_vectors}")
print(f"Reflectivity: {my_profile.reflectivity}")
print(f"Errors on the reflectivity: {my_profile.reflectivity_e}")

Theta values: [0.045225 0.047505 0.049755 ... 3.525275 3.570413 3.615548]
Corresponding q-vectors: [0.01     0.010504 0.011002 ... 0.779022 0.788984 0.798945]
Reflectivity: [0.793168 0.80557  0.817332 ... 0.040644 0.040322 0.039764]
Errors on the reflectivity: [0.042844 0.043178 0.043492 ... 0.009698 0.00966  0.009593]

Let’s plot this!

import plotly.graph_objects as go

# The following two lines are just required for the generation of this document.
import plotly
from plotly.offline import iplot
plotly.offline.init_notebook_mode()

def plot_xrr_curve(islatu_profile: Profile, title: str, log=True):
    """
    Convenience function for plotting simple XRR curves.
    """
    fig = go.Figure().update_layout(title=title,
                                    xaxis_title='Q/Å', yaxis_title='R')
    fig.add_trace(go.Scatter(x=(islatu_profile.q_vectors),
                             y=(islatu_profile.reflectivity), error_y={
        "type": 'data',
        "array": (islatu_profile.reflectivity_e),
        "visible": True},
        name="Islatu"))

    if log:
        fig.update_yaxes(type="log")

    iplot(fig)

title = "Uncorrected profile"
plot_xrr_curve(my_profile, title)

The above curve is pretty much nonsense, the sole exception being that the x-axis is correct. Before trying to perform any corrections, it is worth understanding more about the profile object that islatu has created for us.

Firstly, just as the real experiment was made up from a series of scans, so is the islatu profile object.

print(my_profile.scans)

[<islatu.scan.Scan2D object at 0x137fc7e50>, <islatu.scan.Scan2D object at 0x147767820>, <islatu.scan.Scan2D object at 0x137fde1f0>, <islatu.scan.Scan2D object at 0x147562d30>, <islatu.scan.Scan2D object at 0x147838be0>, <islatu.scan.Scan2D object at 0x147510430>, <islatu.scan.Scan2D object at 0x1472aedc0>, <islatu.scan.Scan2D object at 0x147734e50>]

Each of these scans has its own q_vectors, reflectivities, and reflectivities_e, just as the overall profile does.

first_scan = my_profile.scans[0]

print(f"Theta values: {first_scan.theta}")
print(f"Corresponding q-vectors: {first_scan.q_vectors}")
print(f"Reflectivity: {first_scan.reflectivity}")
print(f"Errors on the reflectivity: {first_scan.reflectivity_e}")

Theta values: [0.045225 0.047505 0.049755 ... 0.15377  0.156033 0.158291]
Corresponding q-vectors: [0.01     0.010504 0.011002 ... 0.034002 0.034502 0.035002]
Reflectivity: [0.940042 0.95474  0.96868  ... 0.018563 0.017533 0.016332]
Errors on the reflectivity: [0.050778 0.051173 0.051545 ... 0.007135 0.006935 0.006693]

Scans also have an attribute called metadata. This contains all of the metadata stored in that scan’s .nxs file. Islatu uses this object to expose several useful pieces of information, such as background and signal regions of interest, the distance between the detector and the sample, and much more (shown below).

example_metadata = first_scan.metadata

print([attr for attr in dir(example_metadata) if not attr.startswith('_')])

['background_regions', 'default_axis', 'default_axis_name', 'default_axis_type', 'default_nxdata', 'default_nxdata_name', 'default_signal', 'default_signal_name', 'detector', 'detector_distance', 'detector_name', 'entry', 'excalibur_detector', 'instrument', 'local_data_path', 'local_path', 'nxfile', 'probe_energy', 'signal_regions', 'src_path', 'transmission']

Each of the scans in a profile contain detector images. Let’s visualize the first image taken in the first scan.

# Image data is stored as a numpy array, so we'll want to load numpy to
# manipulate the data.
import numpy as np

first_image = first_scan.images[0]

# Because of how the heatmap will be displayed, to make x pixels lie along the
# x-axis we need to take the transpose of our data.
array_transpose = np.transpose(first_image.array)
# Let's also take the log of our data to make it easier to visualize.
array_log_transpose = np.log(array_transpose+0.1)

# We'll want a simple function for plotting heatmaps.
def plot_heatmap(array_to_plot, title):
    """
    Simple function that we can use to plot heatmaps of detector frames.
    """
    fig = go.Figure().update_layout(title=title,
                                    xaxis_title='x-pixels',
                                    yaxis_title='y-pixels')
    fig.add_trace(go.Heatmap(z=array_to_plot, colorscale='Jet'))
    iplot(fig)

# Now plot the image's array.
plot_heatmap(array_log_transpose, "Uncropped detector frame")

As we can see, the signal is localized in the detector. There is not much point carrying out corrections on the whole image, so we should crop our images around the signal. While we could use the above image to manually specify a region of interest, this is usually given by the first region of interest specified in GDA for experiments on I07. So we can carry out the cropping rather neatly and automatically as follows.

from islatu.cropping import crop_to_region

signal_region, = example_metadata.signal_regions
my_profile.crop(crop_to_region, region=signal_region)

# Now lets re-visualize what we have!

array_transpose = np.transpose(first_image.array)
plot_heatmap(array_transpose, "Cropped detector image")

Clearly, the above images will be more efficient to work with. Now it is time to start applying corrections to our reflectivity curve.

The first substantial improvement to the reflectivity and its errors can be made by subtracting background. This replaces the not-so-meaningful default signal stored in the .nxs file with a proper background-subtracted integrated region of interest. Two ways of subtracting background using islatu are shown below.

%%capture
from islatu.background import roi_subtraction
from islatu.region import Region

# Islatu refers to regions of interest using instances of its Region class. For
# these data, I happen to know that a reasonable estimate of the background can
# be made by looking at counts in the following region.
background_region = Region(x_start=1258, x_end=1308, y_start=206, y_end=224)

# Now, background can be subtracted as follows.
my_profile.bkg_sub(roi_subtraction, list_of_regions=[background_region])

# Typically, during reflectometry experiments the I07 beamline, one specifies a
# number of regions of interest in GDA. While the first region usually
# specifies the signal region, other regions of interest indicate background
# regions. If this was done during your experiment, then background can be
# subtracted without hardcoding a background region by uncommenting the
# following line.

# my_profile.bkg_sub(roi_subtraction,
#     list_of_regions=example_metadata.background_regions)

Now let’s see how this affected our profile’s reflectivity curve.

title = "Background subtracted profile"
plot_xrr_curve(my_profile, title)

The first thing to notice is that the uncertainties on each point have substantially decreased. Now that the background has been subtracted, each data point is much more physically meaningful and uncertainties have been correctly calculated (replacing the nexus file defaults). These uncertainties might seem shockingly small, but the simple fact of the matter is that statistical errors in synchrotron measurements are tiny!

To continue, lets make this look a little more… connected. The reason the XRR curve is so bumpy at the moment is that the beam attenuation is changing with scan number. Correcting for attenuation variation between scans will help a lot!

The beam attenuation is stored in the .nxs file that we used to load our profile, so islatu already knows about the attenuation as a function of scan number. All we need to do is tell islatu to correct for it, and we’re done.

my_profile.transmission_normalisation()

# Let's visualize the profile now!
plot_xrr_curve(my_profile, "Transmission + bkg corrected")

This already looks far better. We can flatten the curve above the total external reflection point by applying a footprint correction. Footprint corrections account for the fact that, at small Q, only a small fraction of the beam is incident on the sample surface, which affects reflected intensity. To correct for this, all we need to do is specify the beam FWHM and the sample size.

As mentioned earlier, this is a water sample. In I07, the trough is 20cm long. All distances should be input as SI units.

beam_FWHM = 100e-6
sample_size = 200e-3

my_profile.footprint_correction(beam_width=beam_FWHM, sample_size=sample_size)

# Now plot the resultant curve!
plot_xrr_curve(my_profile, title="Footprint + transmission + bkg corrected")

Next, we should correct for the variation in the intensity of the incident beam when using I07’s double-crystal deflector (DCD) setup. If you didn’t use the DCD in your experiment, don’t worry – simply ignore this step.

When using the DCD, a normalisation file is acquired. You will need to have downloaded this along with your data. This is usually a .dat file. To carry out the DCD normalisation, islatu will need to generate an interpolator from your .dat file.

from islatu.corrections import get_interpolator
from islatu.io import i07_dat_to_dict_dataframe

path_to_DCD_dat_file = data_dir + "404863.dat"

itp = get_interpolator(path_to_DCD_dat_file, i07_dat_to_dict_dataframe)
my_profile.qdcd_normalisation(itp)

# Now check out the plot after correcting for variation in the intensity of the
# incident beam in the DCD setup.

plot_xrr_curve(my_profile, "DCD + footprint + transmission + bkg")

Finally, there are regions of this curve in which data points from different scans are overlappping. Propagating errors correctly, these data can be combined in a continuum of different ways. Islatu can convert this into a smooth reflectivity curve by rebinning your data into linearly (or, optionally, logarithmically) separated bins in q-space, combining data points by an inverse-variance weighted mean (the optimal way of combining data).

number_of_lin_spaced_qs = 4000
my_profile.rebin(number_of_q_vectors=number_of_lin_spaced_qs)

# Now visualize the fully corrected curve!
plot_xrr_curve(my_profile, "Fully corrected reflectivity profile")