Semi-automated scanning procedure

[blazeiro]

Is your feature request related to a problem? Please describe.
During my thesis project I have developed some metrics for HGCROC evaluation, with a focus on the pedestal, ADC, TOA and TOT responses. To make things easier, I wrote a Python script (full_scan.py) that runs multiple pftool TASKS in a given sequence, allowing me to simply leave the scans to run overnight. Not only is this convenient, it would also provide some standardised procedure for HGCROC performance comparison (at least with Lund).
I wanted to translate this script into something more pflib-compatible, but I am not sure whether this is possible. Currently, the script calls various TASKS and inputs the necessary settings, which are read with pflib::readline_int. This can be done via the Python os.system function, launching pftool with a specific set of pre-written commands.
I am unsure how to replicate this within pftool, i.e. using a c++ pftool TASK that calls other TASKS.

Describe the solution you'd like
Ideally, the script would be a simple pftool TASK/feature that calls other TASKS in a specific sequence. This could be broken into chunks, which could then be called separately, like in the original Python script. The input settings for all TASKS would be filled in at the beginning.

Describe alternatives you've considered
Initially, I thought it may be possible to use cout to fill in the initial settings for each TASK, but I don't think that's how it works. I also considered changing all the relevant TASKS to fit the procedure, but I would like to avoid that. In the case that this whole endeavour is fruitless, I will describe the procedure in the wiki.

Additional context
For more context, I primarily used PARAMETER_TIMESCAN and CHANNEL_WISE_CALIB_SCAN for ADC/TOA/TOT data acquisition. These are taken over all channels of the chip and across a wide CALIB range, which takes probably about 8h. Using the script I was able to scan one chip a day. You can find some of the results in the new wiki entries for data analysis scripts!

[tomeichlersmith]

The pftool script mechanism is probably the easiest way to do this right now (although, the script is not a .yaml file, it is just a list of text that is entered into pftool as if it was typed. I recommend using the extension .txt or .pftool to avoid confusion).

Technically, the tasks are all just functions that could be called in sequence (as you probably already know), so calling several of them in sequence within the C++ is "easy" but does not get around the interaction necessary to enter details into the tasks as they run. The interaction is mediated by readline, a C library for getting keyboard input.

I don't think we'd want to do the TTY hijinks necessary to "pre-fill" the content that readline would extract. Instead, I think the work of defining this procedure would need to be translated into updating the code for these tasks themselves, extracting the automated logic from the user interaction. My initial idea was that each of these "steps" in the procedure would be an "algorithm" (app/tool/algorithm) which never has any user interaction that can be called individually or in sequence within different "tasks" (app/tool/tasks). I haven't told anyone this idea explicitly, so the implementation has certainly drifted away from this design, but I still think it could work. Does that make sense?

[blazeiro]

To quickly address the first part, if the script that is "not a .yaml file" you're referring to is the loaded sequence of text commands within the larger Python script, then yes, I would be happy to change that to .txt or .pftool. I was under the impression that pftool -s [command list] would only work with a .yaml extension, so this good to hear (since it was confusing to me as well).

When it comes to the translation of the Python script to pflib, I'm not quite sure I understand the idea. Do you mean that we'd need to update each of the called tasks, such that user input is detached from them and put into separate algorithm "input-managers"? Or do you mean to have the "step" algorithms be responsible for calling the relevant tasks, in which case I still don't understand how we'd get the user-input variables without stopping at each new task.

[tomeichlersmith]

The user input would be detached and left within app/tool/tasks. The app/tool/algorithm is where the "step" algorithms that do the logic of tuning would reside. These algorithms would have their required inputs be arguments to the function and the arguments to those functions would either be input by the user (as part of a single-step task) or deduced in some other way (as part of a multi-step task).