<br><font size=2 face="sans-serif">Hi,</font>
<br>
<br><font size=2 face="sans-serif">I've got through padb test now (assuming jobid is just numeric part), the result is OK, and I met 2 bugs:</font>
<br>
<br><font size=2 face="sans-serif">1- Path of remote padb:</font>
<br>
<br><font size=2 face="sans-serif">[thipa@xn5 padb_open]$ ./padb -O rmgr=pbs -tx 27611</font>
<br><font size=2 face="sans-serif">einner: xn20: bash: <b>./padb: No such file or directory</b></font>
<br><font size=2 face="sans-serif">einner: pdsh@xn5: xn20: ssh exited with exit code 127</font>
<br><font size=2 face="sans-serif">einner: xn19: bash: <b>./padb: No such file or directory</b></font>
<br><font size=2 face="sans-serif">einner: pdsh@xn5: xn19: ssh exited with exit code 127</font>
<br><font size=2 face="sans-serif">Unexpected EOF from Inner stdout (connecting)</font>
<br><font size=2 face="sans-serif">Unexpected EOF from Inner stderr (connecting)</font>
<br><font size=2 face="sans-serif">Waiting for signon from 2 hosts.</font>
<br><font size=2 face="sans-serif">Unexpected exit from parallel command (state=connecting)</font>
<br><font size=2 face="sans-serif">[thipa@xn5 padb_open]$</font>
<br>
<br><font size=2 face="sans-serif">[thipa@xn20 padb_open]$ ssh -V</font>
<br><font size=2 face="sans-serif">OpenSSH_5.1p1, OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008</font>
<br>
<br><font size=2 face="sans-serif">I think from OpenSSH_5.1, the remote SSH does not execute .bashrc anymore</font>
<br><font size=2 face="sans-serif">So PATH variable is not set from your .bashrc,</font>
<br><font size=2 face="sans-serif">As the consequence path is not found,</font>
<br><font size=2 face="sans-serif">So path to remote host must be a full path, I did the patch as follows:</font>
<br>
<br><font size=2 face="sans-serif">sub pbs_setup_job {</font>
<br>
<br><font size=2 face="sans-serif">.</font>
<br><font size=2 face="sans-serif">.</font>
<br>
<br><font size=2 face="sans-serif"> my $pwd=$ENV{PWD};</font>
<br><font size=2 face="sans-serif"> my $dirnm = dirname ($0);</font>
<br><font size=2 face="sans-serif"> my $base = basename ($0);</font>
<br><font size=2 face="sans-serif"> # if padb is launch as padb then dirname is .</font>
<br><font size=2 face="sans-serif"> # if padb is launched with a full path then dir is full</font>
<br><font size=2 face="sans-serif"> my $out;</font>
<br><font size=2 face="sans-serif"> if ($dirnm eq ".") {</font>
<br><font size=2 face="sans-serif"> $out=" $pwd\/$base ";</font>
<br><font size=2 face="sans-serif"> } else {</font>
<br><font size=2 face="sans-serif"> $out=" $0 ";</font>
<br><font size=2 face="sans-serif"> }</font>
<br><font size=2 face="sans-serif"> $pcmd{padb_path} = $out;</font>
<br>
<br><font size=2 face="sans-serif">.</font>
<br><font size=2 face="sans-serif">}</font>
<br>
<br><font size=2 face="sans-serif">sub go_job {</font>
<br>
<br><font size=2 face="sans-serif">.</font>
<br>
<br><font size=2 face="sans-serif">my $padb_path = $pcmd{padb_path};</font>
<br>
<br><font size=2 face="sans-serif">.</font>
<br><font size=2 face="sans-serif">.</font>
<br><font size=2 face="sans-serif"> #replace this line $cmd .= " $0 --inner"; by</font>
<br>
<br><font size=2 face="sans-serif"> if (!defined $padb_path) {</font>
<br><font size=2 face="sans-serif"> $cmd .= " $0 --inner";</font>
<br><font size=2 face="sans-serif"> } else {</font>
<br><font size=2 face="sans-serif"> $cmd .= " $padb_path --inner ";</font>
<br><font size=2 face="sans-serif"> }</font>
<br>
<br><font size=2 face="sans-serif">.</font>
<br>
<br><font size=2 face="sans-serif">.</font>
<br>
<br><font size=2 face="sans-serif">}</font>
<br>
<br><font size=2 face="sans-serif">If you have another idea i take it.</font>
<br>
<br>
<br><font size=2 face="sans-serif">2- Use of uninitialized value in subtraction (-) at ./padb line 4077</font>
<br>
<br><font size=2 face="sans-serif">[thipa@xn5 padb_open]$ ./padb -O rmgr=pbs -O stack-shows-locals=no -O stack-shows-params=no -tx 27611</font>
<br><font size=2 face="sans-serif"><b>Use of uninitialized value in subtraction (-) at ./padb line 4077</b>.</font>
<br><font size=2 face="sans-serif">-----------------</font>
<br><font size=2 face="sans-serif">[0-1,3,6] (4 processes)</font>
<br><font size=2 face="sans-serif">-----------------</font>
<br><font size=2 face="sans-serif">main() at pp_sndrcv_spbl.c:50</font>
<br><font size=2 face="sans-serif"> PMPI_Finalize() at ?:?</font>
<br><font size=2 face="sans-serif"> MPID_Finalize() at ?:?</font>
<br><font size=2 face="sans-serif"> MPIDI_CH3_Progress_wait() at ?:?</font>
<br><font size=2 face="sans-serif"> MPIDU_Sock_wait() at ?:?</font>
<br><font size=2 face="sans-serif"> poll() at ?:?</font>
<br><font size=2 face="sans-serif">-----------------</font>
<br><font size=2 face="sans-serif">[2,4] (2 processes)</font>
<br><font size=2 face="sans-serif">-----------------</font>
<br><font size=2 face="sans-serif">ThreadId: 1</font>
<br><font size=2 face="sans-serif"> -----------------</font>
<br><font size=2 face="sans-serif"> [2] (1 processes)</font>
<br><font size=2 face="sans-serif"> -----------------</font>
<br><font size=2 face="sans-serif"> main() at pp_sndrcv_spbl.c:46</font>
<br><font size=2 face="sans-serif"> PMPI_Recv() at ?:?</font>
<br><font size=2 face="sans-serif"> MPID_Progress_wait() at ?:?</font>
<br><font size=2 face="sans-serif"> MPIDI_CH3_Progress_wait() at ?:?</font>
<br><font size=2 face="sans-serif"> MPIDU_Sock_wait() at ?:?</font>
<br><font size=2 face="sans-serif"> poll() at ?:?</font>
<br><font size=2 face="sans-serif"> ThreadId: 2</font>
<br><font size=2 face="sans-serif"> start_thread() at ?:?</font>
<br><font size=2 face="sans-serif"> fd_server() at server.c:354</font>
<br><font size=2 face="sans-serif"> select() at ?:?</font>
<br><font size=2 face="sans-serif"> -----------------</font>
<br><font size=2 face="sans-serif"> [4] (1 processes)</font>
<br><font size=2 face="sans-serif"> -----------------</font>
<br><font size=2 face="sans-serif"> main() at pp_sndrcv_spbl.c:50</font>
<br><font size=2 face="sans-serif"> PMPI_Finalize() at ?:?</font>
<br><font size=2 face="sans-serif"> MPID_Finalize() at ?:?</font>
<br><font size=2 face="sans-serif"> MPIDI_CH3_Progress_wait() at ?:?</font>
<br><font size=2 face="sans-serif"> MPIDU_Sock_wait() at ?:?</font>
<br><font size=2 face="sans-serif"> poll() at ?:?</font>
<br><font size=2 face="sans-serif"> ThreadId: 2</font>
<br><font size=2 face="sans-serif"> start_thread() at ?:?</font>
<br><font size=2 face="sans-serif"> fd_server() at server.c:354</font>
<br><font size=2 face="sans-serif"> select() at ?:?</font>
<br><font size=2 face="sans-serif">[thipa@xn5 padb_open]$ </font>
<br>
<br><font size=2 face="sans-serif">line 4077 looks like:</font>
<br>
<br><font size=2 face="sans-serif">4058 sub check_signon {</font>
<br><font size=2 face="sans-serif">4059 my ( $comm_data, $data ) = @_;</font>
<br><font size=2 face="sans-serif">4060 return if ( $conf{check_signon} eq 'none' );</font>
<br><font size=2 face="sans-serif">.</font>
<br><font size=2 face="sans-serif">.</font>
<br><font size=2 face="sans-serif">4075 my $rng = rng_create_empty();</font>
<br><font size=2 face="sans-serif">4076 </font>
<br><font size=2 face="sans-serif"><b>4077</b> foreach my $proc ( 0 .. $comm_data->{nprocesses} - 1 ) {</font>
<br><font size=2 face="sans-serif">4078 if ( not defined $here{$proc} ) {</font>
<br><font size=2 face="sans-serif">4079 rng_add_value( $rng, $proc );</font>
<br><font size=2 face="sans-serif">4080 }</font>
<br><font size=2 face="sans-serif">4081 }</font>
<br>
<br><font size=2 face="sans-serif">3- Question about starting inner padb:</font>
<br>
<br>
<br><font size=2 face="sans-serif">How can I start an inner padb by hand on a remote host to debug such as:</font>
<br><font size=2 face="sans-serif">perl -d ./padb --inner --jobid=27611.xn0 --stack-trace -O rmgr="pbs" --line-formatted</font>
<br><font size=2 face="sans-serif">like I did it before, because this command doesn't work anymore.</font>
<br><font size=2 face="sans-serif">You have changed it with "call back" and communication on ports.</font>
<br>
<br>
<br><font size=2 face="sans-serif">Here is the diff again r311 (diff r311 newone).</font>
<br>
<br>
<br>
<br><font size=2 face="sans-serif">So you can integrate my new patch and try to correct the point 2,</font>
<br><font size=2 face="sans-serif">and send me back the new one, i will test it over.</font>
<br>
<br><font size=2 face="sans-serif">Regards,</font>
<br><font size=2 face="sans-serif">Thipadin.</font>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<table width=100%>
<tr valign=top>
<td>
<td><font size=1 face="sans-serif"><b>Ashley Pittman <ashley@pittman.co.uk></b></font>
<p><font size=1 face="sans-serif">11/18/2009 09:00 AM</font>
<br>
<td><font size=1 face="Arial"> </font>
<br><font size=1 face="sans-serif"> Pour : thipadin.seng-long@bull.net</font>
<br><font size=1 face="sans-serif"> cc : florence.vallee@bull.net, francois.wellenreiter@bull.net, padb-devel@pittman.org.uk, Sylvain.JEAUGEY@bull.net</font>
<br><font size=1 face="sans-serif"> Objet : Re: Réf. : Re: [padb-devel] Patch for Support of PBS Pro resource manager</font></table>
<br>
<br><font size=2 face="Courier New">On Tue, 2009-11-17 at 17:17 +0100, thipadin.seng-long@bull.net wrote:<br>
> I have to break on the screen to get prompt: <br>
> So I guess it is a infinite loop. <br>
> I have changed 'while(@output)' for 'foreach(@output)', to correct<br>
> this probleme.<br>
<br>
That looks like a simple mistake on my part. I prefer not to use $_ in<br>
my code (either implicitly or explicitly) as I think it makes it less<br>
readable, once the code works it's easy enough to make variables<br>
explicit however, I just didn't do it in the patch because I prefer to<br>
change as least as I can possibly get away with unless I can test it<br>
immediately.<br>
<br>
> 2- Job is not found: <br>
> <br>
> So when the loop is disappeared i can go further: <br>
> <br>
> ./padb -O rmgr=pbs -tx 27611.xn0 <br>
> Job 27611.xn0 is not active <br>
> [thipa@xn5]$ qstat <br>
> Job id Name User Time Use S Queue <br>
> ---------------- ---------------- ---------------- -------- - ----- <br>
> 27611.xn0 STDIN thipa 00:00:06 R workq<br>
<br>
> <br>
> [thipa@xn5]$ <br>
> <br>
> The jobs that are display by qstat have the suffice with .xn0 (which<br>
> is the server), <br>
> so we used to pick up the whole job id as input jobid. <br>
> So something have to be changed (code or synopsis).<br>
<br>
I guess the job id as you requested it (27611.xn0) does not match what<br>
it returned by pbs_get_jobs(), there is a problem here to do with the<br>
server. In the past all job id's have been numeric, this hasn't been a<br>
problem but isn't something that I've strived for, it's just that so far<br>
all resource managers have worked that way so that's how I think of it.<br>
There is no technical reason for this to be true however so how about we<br>
just say that in the future jobid's have to be alphanumeric strings,<br>
this would work in this case although would have the downside you<br>
couldn't specify the job as 27611 in the case above.<br>
<br>
padb --show-jobs should show you what padb thinks the job id's are and<br>
of course using -a rather than specifying a job tells it to use all jobs<br>
so it'll just attempt to target one in the case above, regardless of<br>
what it thinks it's called.<br>
<br>
I'd be happy for a patch supporting either implementation, i.e. I don't<br>
have a strong preference either way. You can either have the jobid<br>
encompass both the number and the server or you could continue with what<br>
I attempted to encode in the patch I sent you where the job id is the<br>
number and the server becomes a configuration option.<br>
<br>
Actually this could make life easier for slurm and the way it handles<br>
job steps, it effectively appends ".0" to the padb job id before handing<br>
it over to slurm so this could probably be simplified if the .0 became a<br>
optional part of the job id itself rather than a separate configuration<br>
option.<br>
<br>
> I am waiting for your patch (or reply) to continue.<br>
<br>
I hope this helps you along the way, I can't really code anything from<br>
here as I don't have access to a pbs system.<br>
<br>
Ashley,<br>
<br>
-- <br>
<br>
Ashley Pittman, Bath, UK.<br>
<br>
Padb - A parallel job inspection tool for cluster computing<br>
http://padb.pittman.org.uk<br>
<br>
</font>
<br>
<br>