BIDS Tutorial Series: HeuDiConv Walkthrough

May 17, 2018

Introduction

Welcome back to the next tutorial in the tutorial series “Getting Started with BIDS”. In this tutorial, we will be utilizing an off-the-shelf solution called HeuDiConv. This tutorial will illustrate a detailed step-by-step guide on how to use HeuDiConv. The guide will take one subject and iteratively add one session at a time until the dataset is organized and validated. This part will provide separate command line inputs to run HeuDiConv. We will be using DICOMs from the MyConnectome dataset found here and following the BIDS Specification version 1.1.0. Another tutorial has described using HeuDiConv on a different dataset. The next part of this tutorial series will examine another off-the-shelf solution to consider using to convert your dataset into a validated BIDS dataset.

Table of Contents

  1. Setting up your environment
  2. Run HeuDiConv on ses-001 scans to get the dicominfo file
  3. Examine ses-001 dicominfo file to generate heuristic.py
  4. Run HeuDiConv on ses-001 scans with heuristic.py and validate
  5. Run HeuDiConv on ses-005 scans to get the dicominfo file
  6. Examine ses-005 dicominfo file and update heuristic.py
  7. Run HeuDiConv on ses-005 scans with updated heuristic.py and validate
  8. Run HeuDiConv on ses-025 scans to get the dicominfo file
  9. Examine ses-025 dicominfo file and update heuristic.py
  10. Run HeuDiConv on ses-025 scans with updated heuristic.py and validate

 

2A. Guide to using HeuDiConv

Step 1. To begin, we first need to initialize the environment. We will be utilizing Docker for running HeuDiConv. After one has gotten Docker, we can use the command line to download HeuDiConv. The command is shown below. Once that has downloaded, one may begin the conversation

docker pull nipy/heudiconv:latest

We will be starting with converting ses-001 within sub-01.

dicom_1struc

Step 2. We are ready to run HeuDiConv. The command syntax can be seen below. Let’s break down this syntax. The ‘docker run –rm -it’ is calling docker. The ‘-v /Users/franklinfeingold/Desktop/HeuDiConv_walkthrough:/base’ is mounting this path and calling it base. This means that base can be used instead of the entire path. The ‘nipy/heudiconv:latest’ is calling the latest version of HeuDiConv. The -d is the dicom directory. Notice we started the path with /base instead of the full path and subject and session are curly bracketed, this is because they are defined later in the call as -s for subject and -ss for session. The -o is where the .heudiconv directory will be, -f is for the heuristic file, and -c is the converter to use.

docker run --rm -it -v /Users/franklinfeingold/Desktop/HeuDiConv_walkthrough:/base nipy/heudiconv:latest -d /base/Dicom/sub-{subject}/ses-{session}/SCANS/*/DICOM/*.dcm -o /base/Nifti/ -f convertall -s 01 -ss 001 -c none --overwrite

This command will output a hidden .heudiconv folder within the Nifti folder that contains a dicominfo .tsv file used to construct the heuristic file.

Step 3. Now copy the dicominfo .tsv file out of the hidden .heudiconv folder. We temporarily placed this folder at the same level as Dicom and Nifti. This file is used to construct the heuristic.py file.

cp /Users/franklinfeingold/Desktop/HeuDiConv_walkthrough/Nifti/.heudiconv/01/info/dicominfo_ses-001.tsv /Users/franklinfeingold/Desktop/HeuDiConv_walkthrough

heudiconv_tsv_ses1

We can open the dicominfo file and evaluate the different columns. Some of the different columns are pictured below. This information will be used to create the heuristic.py file. The heuristic.py file is used to convert and organize the data. Here is an example of how a complete one may look. Our heuristic.py can be seen below. Once the heuristic file has been generated, one may delete the dicominfo .tsv file. Please place the heuristic.py file in a code folder within Nifti, pictured below. 

tsvfile_ses1

import os
def create_key(template, outtype=('nii.gz',), annotation_classes=None):
    if template is None or not template:
        raise ValueError('Template must be a valid format string')
    return template, outtype, annotation_classes
def infotodict(seqinfo):
    """Heuristic evaluator for determining which runs belong where
    allowed template fields - follow python string module:
    item: index within category
    subject: participant id
    seqitem: run number during scanning
    subindex: sub index within group
    """
    t1w = create_key('sub-{subject}/{session}/anat/sub-{subject}_{session}_run-00{item:01d}_T1w')

    info = {t1w: []}
    
    for idx, s in enumerate(seqinfo):
        if (s.dim1 == 320) and (s.dim2 == 320) and ('t1_fl2d_tra' in s.protocol_name):
            info[t1w].append(s.series_id)
    return info

struc_code_heuristic

Step 4. Now that we have the heuristic.py script we can run HeuDiConv again, but changing -c from none to dcm2niix and adding -b. The call is seen below.

docker run --rm -it -v /Users/franklinfeingold/Desktop/HeuDiConv_walkthrough:/base nipy/heudiconv:latest -d /base/Dicom/sub-{subject}/ses-{session}/SCANS/*/DICOM/*.dcm -o /base/Nifti/ -f /base/Nifti/code/heuristic.py -s 01 -ss 001 -c dcm2niix -b --overwrite

Once HeuDiConv completes running, the output is pictured below. We are ready to validate this dataset.

ses1_outputvalidate_1

We find this is a valid BIDS dataset.

Bids valid dataset

Step 5. We will now be adding in ses-005. 

dicom_struc5

Similar to the approach taken for converting and organizing the ses-001 images, we will be running HeuDiConv to get the dicominfo .tsv file. To do this, the syntax will follow closely to the call used for getting the ses-001 dicominfo file.

docker run --rm -it -v /Users/franklinfeingold/Desktop/HeuDiConv_walkthrough:/base nipy/heudiconv:latest -d /base/Dicom/sub-{subject}/ses-{session}/SCANS/*/DICOM/*.dcm -o /base/Nifti/ -f convertall -s 01 -ss 005 -c none --overwrite

This will output the dicominfo .tsv file for ses-005. Please copy the dicominfo .tsv file out of the .heudiconv directory.

Step 6. We can begin adding to the heuristic.py file to convert and organize the diffusion scans. To do this, we look at the dicominfo.tsv file and determine the condition to organize the scan. This will be added to the heuristic.py file. 

dicominfo_5

import os
def create_key(template, outtype=('nii.gz',), annotation_classes=None):
    if template is None or not template:
        raise ValueError('Template must be a valid format string')
    return template, outtype, annotation_classes
def infotodict(seqinfo):
    """Heuristic evaluator for determining which runs belong where
    allowed template fields - follow python string module:
    item: index within category
    subject: participant id
    seqitem: run number during scanning
    subindex: sub index within group
    """
    t1w = create_key('sub-{subject}/{session}/anat/sub-{subject}_{session}_run-00{item:01d}_T1w')
    func_rest = create_key('sub-{subject}/{session}/func/sub-{subject}_{session}_task-rest_run-00{item:01d}_bold')

    info = {t1w: [], func_rest: []}
    
    for idx, s in enumerate(seqinfo):
        if (s.dim1 == 320) and (s.dim2 == 320) and ('t1_fl2d_tra' in s.protocol_name):
            info[t1w].append(s.series_id)
        if (s.dim1 == 128) and (s.dim2 == 128) and ('Resting State fMRI MBEPI' in s.protocol_name):
            info[func_rest].append(s.series_id)
    return info

Step 7. Now that we have the updated heuristic.py file, we can delete the dicominfo.tsv file and run HeuDiConv again to convert and organize ses-005.

docker run --rm -it -v /Users/franklinfeingold/Desktop/HeuDiConv_walkthrough:/base nipy/heudiconv:latest -d /base/Dicom/sub-{subject}/ses-{session}/SCANS/*/DICOM/*.dcm -o /base/Nifti/ -f /base/Nifti/code/heuristic.py -s 01 -ss 005 -c dcm2niix -b --overwrite

We find that HeuDiConv converted and organized the scan, so we are ready to validate our dataset. In addition, shown below is the file structure up to this point.

convert_struc_5validate_5

We find this is a valid BIDS dataset.

Bids valid dataset

Step 8. Now we are ready to add in the ses-025 scans. The Dicom structure is pictured below.

dicom_struc_25

Following the same approach taken for ses-001 and ses-005 conversation and organization, we will run HeuDiConv to get the dicominfo .tsv file. To do this, the command will follow closely to the HeuDiConv call used above.

This will run and output the dicominfo .tsv file. Copy the file out of the .heudiconv directory.

docker run --rm -it -v /Users/franklinfeingold/Desktop/HeuDiConv_walkthrough:/base nipy/heudiconv:latest -d /base/Dicom/sub-{subject}/ses-{session}/SCANS/*/DICOM/*.dcm -o /base/Nifti/ -f convertall -s 01 -ss 025 -c none --overwrite

Step 9. We can determine the organizational rules for each of the scans from the dicominfo file and add them to the heuristic.py file. This follows the same process done for determining the rules to organize the ses-001 and ses-005 scans. We have pictured the dicominfo file and the completed heuristic.py file.

dicominfo_25

 

import os
def create_key(template, outtype=('nii.gz',), annotation_classes=None):
    if template is None or not template:
        raise ValueError('Template must be a valid format string')
    return template, outtype, annotation_classes
def infotodict(seqinfo):
    """Heuristic evaluator for determining which runs belong where
    allowed template fields - follow python string module:
    item: index within category
    subject: participant id
    seqitem: run number during scanning
    subindex: sub index within group
    """
    t1w = create_key('sub-{subject}/{session}/anat/sub-{subject}_{session}_run-00{item:01d}_T1w')
    func_rest = create_key('sub-{subject}/{session}/func/sub-{subject}_{session}_task-rest_run-00{item:01d}_bold')
    func_rest_matrix96 =  create_key('sub-{subject}/{session}/func/sub-{subject}_{session}_task-rest_run-00{item:01d}_bold')
    func_rest_matrix96_sbref =  create_key('sub-{subject}/{session}/func/sub-{subject}_{session}_task-rest_run-00{item:01d}_sbref')
    fmap_mag =  create_key('sub-{subject}/{session}/fmap/sub-{subject}_{session}_magnitude')
    fmap_phase = create_key('sub-{subject}/{session}/fmap/sub-{subject}_{session}_phasediff')
    dwi = create_key('sub-{subject}/{session}/dwi/sub-{subject}_{session}_run-00{item:01d}_dwi')

    info = {t1w: [], func_rest: [],    func_rest_matrix96: [], func_rest_matrix96_sbref: [], fmap_mag: [], fmap_phase: [], dwi: []}
    
    for idx, s in enumerate(seqinfo):
        if (s.dim1 == 320) and (s.dim2 == 320) and ('t1_fl2d_tra' in s.protocol_name):
            info[t1w].append(s.series_id)
        if (s.dim1 == 128) and (s.dim2 == 128) and ('Resting State fMRI MBEPI' in s.protocol_name):
            info[func_rest].append(s.series_id)
        if (s.dim1 == 96) and (s.dim4 == 518) and ('Resting State fMRI MBEPI_matrix96_BW2004' in s.protocol_name):
            info[func_rest_matrix96].append(s.series_id)
        if (s.dim1 == 96) and (s.dim4 == 1) and ('Resting State fMRI MBEPI_matrix96_BW2004' in s.protocol_name):
            info[func_rest_matrix96_sbref].append(s.series_id)
        if (s.dim3 == 136) and (s.dim4 == 1) and ('gre_field_mapping' in s.protocol_name):
            info[fmap_mag] = [s.series_id]
        if (s.dim3 == 68) and (s.dim4 == 1) and ('gre_field_mapping' in s.protocol_name):
            info[fmap_phase] = [s.series_id]
        if (s.dim2 == 128) and (s.dim4 == 64):
            info[dwi].append(s.series_id)
    return info

Step 10. We have updated the heuristic.py file, so we can delete the dicominfo .tsv file and run HeuDiConv again to convert and organize ses-025. The command to do this is seen below.

docker run --rm -it -v /Users/franklinfeingold/Desktop/Heudiconv_walkthrough:/base nipy/heudiconv:latest -d /base/Dicom/sub-{subject}/ses-{session}/SCANS/*/DICOM/*.dcm -o /base/Nifti/ -f /base/Nifti/code/heuristic.py -s 01 -ss 025 -c dcm2niix -b --overwrite

After running HeuDiConv for ses-025, we are ready to validate our dataset. We have pictured below the ses-025 Nifti file structure.

convert_struc_25

upload_25

We find this is a valid BIDS dataset!

Bids valid dataset

This tutorial demonstrated how to utilize HeuDiConv for converting DICOMs into a validated BIDS dataset. In the next part, we will walk through another off-the-shelf solution to consider for converting your datasets.

1 Comment

  1. Dylan 6 days

    Is there a tutorial for using HeuDiConv without Docker? Would be extremely helpful! Thanks in advance.