"--no-ssh" not working ?!?

[jedie]

I start the client on windows with: python.exe Scripts\drqueue --verbose --no-ssh slave

i see the message "Waiting for SSH tunnel to start up . . ."

So:

DrQueueIPython/bin/drqueue

Lines 1015 to 1020 in 3b08a0a

	# wait until port 27017 of MongoDB is available
	mongodb_available = False
	while mongodb_available == False:
	mongodb_available = is_port_open("127.0.0.1", 27017)
	time.sleep(2)
	print("Waiting for SSH tunnel to start up . . . ")

is running?!?

[kaazoo]

The line in

DrQueueIPython/bin/drqueue

Line 242 in 3b08a0a

if (args.no_ssh != True) and (args.func != master_daemon) and (localhost == False):

is responsible.

It should only the SSH tunnel if '--no-ssh' was not requested AND you are running the master AND you are connecting to localhost.

Please provide full verbose output.

[kaazoo]

I tried it out on my machine and did some changes in f7f6f61.

If you start the master via drqueue master -v --no-ssh, it won't fill out the 'ssh' line in IPYTHONDIR/profile_default/security/ipcontroller-engine.json . This will make the slave started via drqueue slave -v not open a SSH tunnel (for both MongoDB and registration to the ipcontroller process inside the master).

If you start the master via drqueue master -v, it will fill out the 'ssh' line with 'user@host'. The slave started by drqueue slave -v will not open a SSH tunnel (for both MongoDB and registration to the ipcontroller process inside the master) when it realizes that it's running on the same machine.

If you want to force the slave to start without SSH, you can also use drqueue slave -v --no-ssh.

Please test with my changes.

[jedie]

First:
I don't want to skip "ssh", really... But it seems just to be a simpler way to start the windows/linux variant.
Setup sshfs on windows seems to be not easy. https://ssl.drqueue.org/redmine/projects/drqueueipython/wiki/SetupAndConfiguration#Windows-7-Python-27 says i have to install four different packages for this...

If you start the master via drqueue master -v --no-ssh, it won't fill out the 'ssh' line in IPYTHONDIR/profile_default/security/ipcontroller-engine.json . This will make the slave started via drqueue slave -v not open a SSH tunnel (for both MongoDB and registration to the ipcontroller process inside the master).

This seems not to work here. I start master with "--no-ssh" but it adds the "ssh" key to "ipcontroller-engine.json" :(

...some minutes later...

I found the problem in parsing the augments!
I add some debug prints:

    print('DEBUG: sys.argv:', sys.argv)
    print('DEBUG: args:', args)

and the output is:

+ drqueue -v --no-ssh master
DEBUG: sys.argv: ['/home/jens/DrQueue_env/bin/drqueue', '-v', '--no-ssh', 'master']
DEBUG: args: Namespace(func=<function master_daemon at 0xb652daac>, no_ssh=False, verbose=False)

So, it can't work. But why are the cmd aguments not parsed?!?

btw. i would suggest to use "click": http://click.pocoo.org/ for the cli! Really nice solution!

EDIT: Found the problem: I started with: drqueue -v --no-ssh master but i have to start with: drqueue master -v --no-ssh !
e.g.:

+ drqueue master -v --no-ssh
DEBUG: sys.argv: ['/home/jens/DrQueue_env/bin/drqueue', 'master', '-v', '--no-ssh']
DEBUG: args: Namespace(func=<function master_daemon at 0xb653daac>, no_ssh=True, verbose=True)

That's bad, isn't it?

[jedie]

OK, after change the cli args with jedie/DrQueueHelperFiles@02c5365 i see some connection activity on master:

==> ipcontroller.log <==
2015-08-04 11:16:05.953 [IPControllerApp] client::client '3b58c1a1-40ab-4f18-b3c9-8c6b0e33c9a4' requested u'registration_request'
2015-08-04 11:16:06.078 [IPControllerApp] WARNING | iopub::IOPub message lacks parent: {'parent_header': {}, 'msg_type': u'status', 'msg_id': u'b39cb24e-91fb-4dee-bd9e-1f3bbfcfb5fb', 'content': {u'execution_state': u'starting'}, 'header': {u'date': datetime.datetime(2015, 8, 4, 11, 16, 7, 557000), u'username': u'username', u'session': u'3b58c1a1-40ab-4f18-b3c9-8c6b0e33c9a4', u'msg_id': u'b39cb24e-91fb-4dee-bd9e-1f3bbfcfb5fb', u'msg_type': u'status'}, 'buffers': [], 'metadata': {}}
2015-08-04 11:16:10.572 [IPControllerApp] registration::finished registering engine 0:3b58c1a1-40ab-4f18-b3c9-8c6b0e33c9a4
2015-08-04 11:16:10.574 [IPControllerApp] engine::Engine Connected: 0

But slave raise immediately a traceback:

ipengine -s c:\users\jens\appdata\roaming\drqueue_env\src\drqueueipython\EGG-INFO\scripts\get_slave_information.py --ssh=
IPython engine started with PID 2656. Logging to Q:\logs\ipengine_192.168.7.12.log.
Traceback (most recent call last):
  File "Scripts\drqueue", line 6, in <module>
    exec(compile(open(__file__).read(), __file__, 'exec'))
  File "C:\Users\jens\AppData\Roaming\DrQueue_env\src\drqueueipython\bin\drqueue", line 1047, in <module>
    main()
  File "C:\Users\jens\AppData\Roaming\DrQueue_env\src\drqueueipython\bin\drqueue", line 272, in main
    args.func(args)
  File "C:\Users\jens\AppData\Roaming\DrQueue_env\src\drqueueipython\bin\drqueue", line 971, in slave_daemon
    os.waitpid(IPENGINE_PID, 0)
OSError: [Errno 10] No child processes

[kaazoo]

Please have a look in get_slave_information.py for any Windows-incompatible stuff.

[jedie]

os.waitpid(IPENGINE_PID, 0)
OSError: [Errno 10] No child processes

Is fixed with: jedie@69cbb61

[kaazoo]

I added your changes: e686d09
However, os.waitpid() is also used a couple of times in the signal handlers of master and slave.