Access denied

[toshaklg]

Hello there! I wonder if anybody had a similar issue and know how it can be fixed.

I use MAJA 4.2.1 from logiciels.cnes.fr, 'Avignon' data downloaded with PEPS and when I make an attempt to run it on the test data by using

(base) anton@kz:~/MAJA_R/bin$ ./startmaja -f folders.txt -t 31TFJ -d 2017-01-01 -e 2017-03-31

i get a permission denied after start of executing the worklplan:

2021-04-20 16:04:00,407 [INFO ] =============This is Start_Maja v4.2.0==============
2021-04-20 16:04:00,412 [INFO ] Detecting input products...
2021-04-20 16:04:00,415 [INFO ] 5 L1C product(s) detected for tile 31TFJ in /home/anton/MAJA_INPUT/31TFJ
2021-04-20 16:04:00,415 [WARNI] No L2A products detected for tile 31TFJ in /home/anton/MAJA_INPUT/31TFJ
2021-04-20 16:04:00,416 [INFO ] Skipping CAMS file detection.
2021-04-20 16:04:00,416 [INFO ] Checking GIPP files
2021-04-20 16:04:00,416 [INFO ] Setting up GIPP folder: /home/anton/MAJA_R/bin/gipp
2021-04-20 16:04:00,422 [INFO ] Searching for DTM
2021-04-20 16:04:00,423 [INFO ] Found DTM: /home/anton/MAJA_R/bin/dtm/S2__TEST_AUX_REFDE2_31TFJ_1001.HDR
2021-04-20 16:04:00,428 [INFO ] GIPP Creation succeeded for SENTINEL2_NATIF
2021-04-20 16:04:00,429 [INFO ] Not enough L1 products available for a BACKWARD mode. Beginning with INIT...
2021-04-20 16:04:00,429 [INFO ] Not enough L1 products available for a BACKWARD mode. Continuing with INIT...
2021-04-20 16:04:00,429 [INFO ] 5 workplan(s) successfully created:
DATE | TILE | MODE | L1-PRODUCT | ADDITIONAL INFO
2017-01-03 10:44:32 | 31TFJ | INIT | S2A_MSIL1C_20170103T104432_N0204_R008_T31TFJ_20170103T104428.SAFE | Init mode - No previous L2
2017-01-10 10:34:11 | 31TFJ | NOMINAL | S2A_MSIL1C_20170110T103411_N0204_R108_T31TFJ_20170110T103407.SAFE | L2 from previous
2017-01-13 10:44:01 | 31TFJ | NOMINAL | S2A_MSIL1C_20170113T104401_N0204_R008_T31TFJ_20170113T104402.SAFE | L2 from previous
2017-01-30 10:32:51 | 31TFJ | NOMINAL | S2A_MSIL1C_20170130T103251_N0204_R108_T31TFJ_20170130T103249.SAFE | L2 from previous
2017-03-14 10:40:11 | 31TFJ | INIT | S2A_MSIL1C_20170314T104011_N0204_R008_T31TFJ_20170314T104411.SAFE | Init mode - No previous L2
Press Enter to continue...
2021-04-20 16:04:03,041 [INFO ] Beginning workplan execution.
2021-04-20 16:04:03,042 [INFO ] Executing workplan #1/5
/bin/sh: 1: /home/anton/MAJA_R/bin: Permission denied
Traceback (most recent call last):
File "/home/anton/work/MAJA_R/lib/python/StartMaja/Start_maja.py", line 555, in
s.run()
File "/home/anton/work/MAJA_R/lib/python/StartMaja/Start_maja.py", line 502, in run
wp.execute(self.maja, self.dtm, self.gipp, self.userconf)
File "/home/anton/work/MAJA_R/lib/python/StartMaja/Chain/Workplan.py", line 187, in execute
return_code = self.launch_maja(maja, wdir=self.wdir, inputdir=self.input_dir, outdir=self.outdir, conf=conf)
File "/home/anton/work/MAJA_R/lib/python/StartMaja/Chain/Workplan.py", line 170, in launch_maja
return FileSystem.run_external_app(maja, args, logfile=logfile, skip_error=self.skip_errors)
File "/home/anton/work/MAJA_R/lib/python/StartMaja/Common/FileSystem.py", line 188, in run_external_app
raise subprocess.CalledProcessError(return_code, cmd)
subprocess.CalledProcessError: Command '/home/anton/MAJA_R/bin -w /home/anton/MAJA_R/bin/work/Start_maja_95cab0a3918c533c6277ce8eb3320592/maja_working_directory --input /home/anton/MAJA_R/bin/work/Start_maja_95cab0a3918c533c6277ce8eb3320592 --output /home/anton/MAJA_INPUT/31TFJ --mode L2INIT -ucs /home/anton/work/MAJA_R/lib/python/StartMaja/userconf --TileId 31TFJ --loglevel PROGRESS' returned non-zero exit status 126.

I checked the rights and it looks like they are not the issue, but I have no idea what it could be then.
Does anyone have any suggestions?

Thanks!

[jerome-colin]

Hi Anton,
Just to be sure: did you checked the permission of the root dir of Maja only ? Does it applies recursively to all sub-directories ?
Could you please send us the content of folder.txt ?
Thanks,
Jerome

[toshaklg]

Oh, right, I totally forgot about posting this info as well, here it is:

1. Content of folder.txt

[Maja_Inputs]
repWork=./work
repGipp=./gipp
repMNT=./dtm
repL1 =/home/anton/MAJA_INPUT
repL2 =/home/anton/MAJA_INPUT
exeMaja=/home/anton/MAJA_R/bin
repCAMS=/path/to/CAMS
[DTM_Creation]
repRAW=./dtm/raw
repGSW=./dtm/gsw

2. The input directory has the following structure:

/home/anton/MAJA_INPUT/31TFJ/S2A_MSIL1C_20170103T104432_N0204_R008_T31TFJ_20170103T104428.SAFE/
... and more .SAFE folders inside 31TFJ

Regarding permissions, I changed them recursively on "Read and Write" for all, but some systemlinks from MAJA directory threw an error that they cannot be changed, so I left them be.

[jerome-colin]

You don't need to set the write permission to all on the maja directory, but 'x' is mandatory for execution.
I strongly suggest you separate your 'jobs' from the root directory of maja. As far as I see it, the dtm and gipp are all created in the 'bin' dir of Maja, which is neither clean nor safe. You may instead create a separated 'maja_jobs' in your home dir where you create your 'dtm' and 'gipp' using startmaja, and point 'repL2' to another directory than 'repL1'. You'd get something like
~/MAJA_INPUT => where you already store your L1C
~/MAJA_OUTPUT => where Maja writes output files
~/MAJA_JOBS => where you prepare your batch runs, dtm, gipp, folder.txt, etc...
~/MAJA_R => where you keep your executable safe with 'rx' for other

To make it even easier, you could add the ../MAJA_R/bin directory in your $PATH variable right from your .bashrc

Hope this helps,
Jerome

[toshaklg]

Thanks, I will post the results here as soon as I am done.

[petket-5]

Hi @toshaklg ,

More certainly, the issue stems from the fact that you need to append /maja to the exeMaja path - You need to reference the full path to the maja executable not just the folder it is located in.

Kind regards,
Peter

[toshaklg]

Thanks @jerome-colin and @petket-5, I changed the folders, downloaded DEM that MAJA asked me to do and now it looks like it works both ways - with full path and with ./startmaja. However, I now encounter the following issue:

OSError: [Errno 28] No space left on device: '/tmp/raw_dem_gcn2zdi3'

but I assume it asks for even more free space, though I have about 40 GB on the hard drive...

[toshaklg]

Yeah, actually, it is either "no space error" or "permission denied" error which just deliberately appear one after another on different machines.

[jerome-colin]

This looks similar to an open issue on version 4.2.5 : https://gitlab.orfeo-toolbox.org/maja/maja/-/issues/229

[toshaklg]

Okay, with some additional help I found the issue: in the folders.txt exeMaja should actually point on maja executable, not the folder.
Instead of

exeMaja=/home/anton/MAJA_R/bin

I need

exeMaja=/home/anton/MAJA_R/bin/maja

In the readme.md here in the repository, it is a bit controversial info, since folders.txt example is accurate but the description below tells to use a folder, rather than executable for maja. I am not saying that this is wrong, but it definitely took me some time to figure out.

Thanks for quick replies here!

[olivierhagolle]

Sorry for the inaccuracy in the documentation, we'll correct the readme !
Thank you for reporting it !
Olivier

[jbrossar]

Hello @toshaklg ,
Thanks for using startmaja !
I think /tmp directory is mounted as a tmpfs filesystem, could you try to empty it before running startmaja?
Don't hesite to report bug directly on the gitlab (https://gitlab.orfeo-toolbox.org/maja/maja/-/tree/develop), now is our main development platform,
Kind regards,
Julie