Shared Memory Publisher

The Shared Memory Publisher is generic tool to populate a shared memory queue with data, typically the tool is used as a data source to test various Data Task components.

Example usage:

% rtctkExampleShmPub -q exampleDataTaskQueue -d 1000 -g 200 -p 100 -r &

fits-file:
queue-name: exampleDataTaskQueue
queue-size: 1000
sample-delay: 1000
print-every: 100
gen-frames: 200
repeat-mode: 1
Generating data
Writing data to shared memory queue

Samples written:  100
Total time to write 100 : 99012 ms
Average frame time: 990.12 ms

Usage is as follows:

% rtctkExampleShmPub -h

Allowed options:
-h [ --help ]                         produce help message
-f [ --fits-file ] arg                fits input file to: if not provided the app will generate
                                      data
-q [ --queue-name ] arg (=default_shm_queue) shm queue name
-s [ --queue-size ] arg (=1000)       size of the queue
-d [ --sample-delay ] arg (=10)       inter-sample delay in ms
-n [ --numa-node ] arg                numa node for shm queue
-p [ --print-every ] arg (=0)         when to print to screen the number of sample written
-g [ --gen-frames ] arg (=100)        Number of frames to generate
-r [ --repeat-mode ]                  Repeat output when all samples are written

Customisation

An abstract base class ShmPub is provided that needs to specialised by developers.

As the concrete TopicType of data being written to SHM varies between consortia, this information must be provided when deriving from the base class. Additionally the exact implementation how data is being generated or retrieved from FITS files is left open to the developer.

Example customisation in child class:

class ExampleShmPubScao : public ShmPub<ScaoLoopTopic> {
    public:
        ExampleShmPubScao(int argc, char * argv[]): ShmPub<ScaoLoopTopic>(argc, argv)
        {
        }

        std::vector<ScaoLoopTopic> ReadFits(std::string filename) override
        {
        }

        std::vector<ScaoLoopTopic> GenData(int num_frames) override
        {
            std::vector<ScaoLoopTopic> vec;
            for(int i=0; i<num_frames; i++)
            {
                ScaoLoopTopic t;
                t.sample_id = i;
                t.wfs.slopes.fill(0);
                t.wfs.intensities.fill(0);
                t.commands.fill(0);
                vec.push_back(t);
            }
            return vec;
        }
};

A utility function to read columns from a provided fits file is also offered in the ShmPub header file. This can be used to read more complex fits files with many columns of data. This will return a vector of the type in the column specified by the name that is passed.

std::vector<T> read_col_from_fits(fitsfile* fptr, char* name, long nrows, bool output=false)

Operation

The primary goal of the ShmPub is to write data into the shared memory buffer with the specified name at a specified rate. This data can then be accessed by other components that know the correct queue name. An example of this can be seen in the exampleDataTask component script.

rtctkExampleShmPub -q exampleDataTaskQueue -d 1000 -g 200 -p 100 -r &

Repeat Mode

Running the ShmPub with option -r enables repeat mode which is thought to be the most common mode of operation. In this mode the shared memory publisher will write into shared memory all the data loaded or generated. When the data has been exhausted it will start again at the beginning of the data set. As not to affect any Data Task reading from the shared memory buffer, the sample_id will always increase. This adds the constraint that the sample_id must be part of any topic being propagated with this tool.

Troubleshooting

If the Shared Memory Publisher tool is not shutdown correctly it may not clean up properly. This can cause problems during the next run.

Typically a shared memory writer creates a file under /dev/shm/ipcq-$(queue_name), if the writer does not terminate gracefully this file may not be deleted and any attempt at creating a new queue with the same name will fail. In this case the file will need to be manually deleted.