geopsy.org

Dear Marc and all,

I just stumbled over the following problem using DinverExt plugin:
The sampler creates hexadress like named subdirectories for each
thread to output the 'parameters' file .
Using the target I usually call then a bash or python script to
read the parameters file into some code for computing the misfit.
Now the problem is that I can't do this anymore as the subdirectory
name creation is done in dinver - I couldn't find whether it is
possible to compute only on a single thread nor how to select
a predefined name for the subdirectories.
Eventually there is now a different philosophy with DinverExt or
how to adapt to the parallelized sampling of the parameter space,
but it is not documented so far. I assume dinver expected the misfit files
being created into the same subdirectories?

Best Regards
Matthias

Hi Matthias,

There were a few bugs around dinver and dinverext in release 3.3.2 (and probably previous also):

• Initialization of Forward object was not correct.
• Try to run a dispersion curve inversion with Ns0=0, you will a get a crash while starting the Neighborhood because at least one model is required to start it.
• The new dinverext is creating several sub-directories inside the working directory to support parallel computation of the forward problem. But the forward processes were not started in the correct sub-directories. Hence no valid model is generated during the Monte-Carlo phase, which in turn led to the crash mentioned above.

This is now fixed for 3.3.3, but meanwhile you can access the git repository for branch 3.3 (version 3.3.3-preview). I tested the fix on the case provided under dinverext/share (rms5) with three parameters from -10 to 10. I add a .dinver file as an example. I tested it through the GUI and with the commande line interface. 3.3.3 is not cleaning the sub-directories correctly as it should. It is implemented for 3.4.0.

To run with a single thread, use option '-j 1', but it does not solve this problem. The philosophy for the end-user remains exactly the same. Only the internal mechanics are affected with geopsy 3.

Best regards,

Marc

Hi Marc,

thanks for checking - I'll probably have to see the example in order
to understand how things work now -
using a shell script in the target I expect a file 'parameters' to be
created in the current working directory as indicated in the target.
The script will read the file parameters calling some forward code
and create a file called 'misfit' which in turn is then read again
in dinver to continue sampling. At least this is how the procedure worked
before. Currently I can't see how I can access the files 'parameters'
in the subdirectories. I could call the forward program multiple times
if I would know in which subdirectories the parameter files are created
and return the corresponding misfit inside those. But most probable
you have a more sophisticated approach taken? It sounds a bit like you are
transferring the call of the script given in target inside the subdirectories
(through linking/pointing the subdirectory into the working directory?)?
In my case I am reading additional control files into the program for
forward computation. Those control files reside in the working directory
given in the target - do you think I would need to specify absolute pathnames
in the script to ensure the correct location of those control files?
Anyway I'll have to update geopsy first, then I'll see whether it works.
Now please - don't work again on this issue until you are back!!
Enjoy Belgium
Best wishes
Matthias

For each inversion thread a sub-directory is created inside the working directory specified in the target options. One file 'parameter' file is created in each sub-directory. The current directory of each sub-process is set to the corresponding sub-directory. Your process must read 'parameter' in its current directory and output 'misfit' also in its current directory. If you access some shared locations, you must ensure that there is no problem to have several simultaneous access. If not run dinver with "-j 1'. There will be only one thread and your common structure will be accessed by only one process at a time.

The names of the sub-directories are unpredictable. It is just the hexadecimal values of the pointers to the Thread structures.

Hi Marc,

yes I got the procedure - let me explain by example

in the target I specify the CWD, let's say CURRDIR,
and some callable script, let's name it 'forward.sh' for convenience.

In forward.sh I call some binary with options, e.g.

myforward -c config.cfg -i input.inp -o misfit

usually input.inp is the file that contains partially the variables
which one wants to optimize for. so before calling the binary
I need to 'prepare' the input.inp using the parameters file spit
out from dinver, from the parallelized NA sampling.
Now the problem is, that I do not know how the subdirectory
is called (e.g. ABEE453D), in which the 'parameters' file is created -
so how can I create my input.inp? this is assuming that the call of the
shell script forward.sh is taking place in CURRDIR.

If dinver is calling the script forward.sh from CURRDIR/ABEE453D
things could work, as the parameters file would be locally available.
also the output file would be created in the correct location for dinver.
However, in this case I would need to call the other input file (config.cfg,
see above) either by adding ../ or specfying an absolute path like
$CURRDIR/config.cfg ...

awk '{ do something here }' parameters > input.inp
$CURRDIR/myforward -c $CURRDIR/config.cfg -i input.inp

would that work then?
Best
Matthias

if CURRDIR is set to $PWD at the beginning of your forward.sh, it should work.