Your message dated Thu, 28 Jul 2016 08:49:53 +0200 with message-id <[email protected]> and subject line Re: Bug#714606: help with fixing ruby-net-ssh: can't add a new key into hash during iteration during ssh.exec has caused the Debian Bug report #714606, regarding ruby-net-ssh: can't add a new key into hash during iteration during ssh.exec to be marked as done.
This means that you claim that the problem has been dealt with. If this is not the case it is now your responsibility to reopen the Bug report if necessary, and/or fix the problem forthwith. (NB: If you are a system administrator and have no idea what this message is talking about, this may indicate a serious mail system misconfiguration somewhere. Please contact [email protected] immediately.) -- 714606: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=714606 Debian Bug Tracking System Contact [email protected] with problems
--- Begin Message ---Package: ruby-net-ssh Version: 1:2.5.2-2 Severity: normal Dear Maintainer, I am currently packaging our own distkeys key distribution ruby script (see #712787 RFS: distkeys/1.0 -- distribute SSH keys). However it only works with Ruby 1.8 for now, as with Ruby 1.9 I get a error back from ruby-net-ssh when trying to add or remove a key: ./distkeys -K somekey.pub -H somehost add Host: somehost Connecting to host somehost (user: ms, port: 9999)... Opening SFTP session... Key somekey added. Creating a backup to .ssh/authorized_keys-2013-07-01.bak if not already done today... Uploading keys to .ssh/authorized_keys-new... File does exist and has correct size, moving to .ssh/authorized_keys... /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:296:in `[]=': can't add a new key into hash during iteration (RuntimeError) from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:296:in `open_channel' from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:320:in `exec' from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:354:in `exec!' from ./distkeys:206:in `block in commit' from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/request.rb:87:in `call' from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/request.rb:87:in `respond_to' from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/session.rb:948:in `dispatch_request' from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/session.rb:911:in `when_channel_polled' from /usr/lib/ruby/vendor_ruby/net/ssh/connection/channel.rb:311:in `call' from /usr/lib/ruby/vendor_ruby/net/ssh/connection/channel.rb:311:in `process' from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:214:in `block in preprocess' from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:214:in `each' from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:214:in `preprocess' from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:197:in `process' from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:161:in `block in loop' from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:161:in `loop' from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:161:in `loop' from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/session.rb:802:in `loop' from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/request.rb:72:in `wait' from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/session.rb:842:in `wait_for' from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/session.rb:320:in `lstat!' from ./distkeys:200:in `commit' from ./distkeys:571:in `handle_host' from ./distkeys:677:in `block in handle_gwhost' from ./distkeys:660:in `each' from ./distkeys:660:in `handle_gwhost' from ./distkeys:692:in `loop' from ./distkeys:797:in `<main>' I also tried after purging ruby-net-ssh which also removes ruby-net-sftp and ruby-net-ssh-gateway and installing as gems: mango:~# gem install net-sftp net-ssh net-ssh-gateway Fetching: net-ssh-2.6.7.gem (100%) Fetching: net-sftp-2.1.2.gem (100%) Successfully installed net-ssh-2.6.7 Successfully installed net-sftp-2.1.2 Successfully installed net-ssh-2.6.7 This gives the following backtrace: /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:299:in `[]=': can't add a new key into hash during iteration (RuntimeError) from /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:299:in `open_channel' from /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:323:in `exec' from /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:357:in `exec!' from ./distkeys:206:in `block in commit' from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/request.rb:87:in `call' from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/request.rb:87:in `respond_to' from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/session.rb:948:in `dispatch_request' from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/session.rb:911:in `when_channel_polled' from /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/channel.rb:311:in `call' from /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/channel.rb:311:in `process' from /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:217:in `block in preprocess' from /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:217:in `each' from /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:217:in `preprocess' from /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:200:in `process' from /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:164:in `block in loop' from /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:164:in `loop' from /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:164:in `loop' from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/session.rb:802:in `loop' from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/request.rb:72:in `wait' from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/session.rb:842:in `wait_for' from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/session.rb:320:in `lstat!' from ./distkeys:200:in `commit' from ./distkeys:571:in `handle_host' from ./distkeys:677:in `block in handle_gwhost' from ./distkeys:660:in `each' from ./distkeys:660:in `handle_gwhost' from ./distkeys:692:in `loop' from ./distkeys:797:in `<main>' With Ruby 1.8 this works. The code where this happens in distkeys as of commit bf13f12e8ca3846998cc1cb610403ad958979377 (I will add a tag with this bug report number) to the git repo: request = @sftp.lstat!(newauthkeyfile) do | response | if response.ok? # File size okay? if response[:attrs].size >= wantedsize puts "File does exist and has correct size, moving to #{@authkeyfile}..." # Move the new keyfile over the old one @ssh.exec!( "mv #{newauthkeyfile} #{@authkeyfile}" ) # We saved the changes, so no unsaved changes anymore @changed = false end end end #@sftp.loop URL to git repo is: git://oss.teamix.org/distkeys.git I am trying to work-around this issue by using sftp.rename now. I think its a better choice than executing the mv command. Unless there is some programming mistake in distkeys that Ruby 1.9 brings to light I bet this is an upstream bug. I am willing to forward / report upstream as well, but first wanted to have this tracked in Debian BTS. ms@mango:~> apt-show-versions | grep ruby-net ruby-net-sftp/wheezy uptodate 1:2.0.5-3 ruby-net-ssh/wheezy uptodate 1:2.5.2-2 ruby-net-ssh-gateway/wheezy uptodate 1.1.0-2 Thanks, Martin -- System Information: Debian Release: 7.1 APT prefers stable APT policy: (500, 'stable'), (350, 'unstable'), (110, 'experimental') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 3.9-1-amd64 (SMP w/4 CPU cores) Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Versions of packages ruby-net-ssh depends on: ii ruby 1:1.9.3 ii ruby1.8 [ruby-interpreter] 1.8.7.358-7 ii ruby1.9.1 [ruby-interpreter] 1.9.3.194-8.1 ruby-net-ssh recommends no packages. ruby-net-ssh suggests no packages. -- no debconf information
--- End Message ---
--- Begin Message ---On 14/04/15 at 15:12 +0200, Martin Steigerwald wrote: > Am Dienstag, 14. April 2015, 14:39:04 schrieb David Suarez: > > Hi, > > > > 2015-04-14 13:00 GMT+02:00 Martin Steigerwald <[email protected]>: > > > Am Dienstag, 14. April 2015, 12:36:33 schrieb Martin Steigerwald: > > >> Cc'ing the bug report as well, feel free to drop the cc for discussion > > >> on mailing list. > > >> an each in session.rb, line 222. > > >> > > >> > > >> Which looks quite central to the working of ruby-net-ssh to me > > >> > > >> # This is called internally as part of #process. It dispatches any > > >> # available incoming packets, and then runs > > >> Net::SSH::Connection::Channel#process > > >> # for any active channels. If a block is given, it is invoked at the > > >> # start of the method and again at the end, and if the block ever > > >> returns > > >> # false, this method returns false. Otherwise, it returns true. > > >> def preprocess > > >> return false if block_given? && !yield(self) > > >> dispatch_incoming_packets > > >> channels.each { |id, channel| channel.process unless > > >> channel.closing? } > > >> return false if block_given? && !yield(self) > > >> return true > > >> end > > >> > > >> > > >> > > >> The calling site inside distkeys is: > > >> > > >> https://github.com/teamix/distkeys/blob/master/distkeys#L174 > > >> > > > > You are right, 'channels' Hash is modified inside 'channels.each' call. > > > > > I am not sure whether it is a work-around or whether it is a valid > > > contraint to be taken into account when using ruby-net-ssh. To me it > > > feels like a work-around, but well… if it works this way. > > > > > > If you still have an idea how to fix it in ruby-net-ssh, please tell me. > > > > Seems like a valid constrain, due that the problem arise when you are > > trying to open a new channel (ssh.exec!) inside the processing block > > of the another channel (sftp.lstat) in the same ssh session. > > > > One fix could be that instead os reusing the actual ssh connection to > > open the sftp one ""@sftp = Net::SFTP::Session.new(@ssh)"", create a > > new sftp connection ""Net::SFTP.start(host, user, options)"". > > Thanks. > > I now just finish the sftp operation before doing the ssh.exec calls and > this appears to work. Just uploaded distkeys-1.1 to github including 1.1 > debian package source. This also fixes the same for the replacing of > authorized_keys with authorized_keys-new which was racy before due to > work-arounding this issue by deleting first and then sftp.rename() which > in current ruby-net-sftp cannot overwrite a file. > > This appears to work just nice. And I also documented this behavior in > the script. > > Will test internally for a while and then probably reapproach with request > for sponsoring for distkeys. Maybe one day will get distkeys into Debian, > I think its quite useful, cause it can put ssh keys to hosts behind > firewalls using ssh port forwarding automatically. After Jessie release :). > > As for this bug, if you think its a valid constraint, feel free to close it. > > I still think it would be good to at least have this documented somewhere > with ruby-net-ssh, cause the interference between ssh and sftp calls are not > that obvious but due to a implementation detail. > > But I can try to send a documentation patch upstream. Hi, I don't think that this is a bug in ruby-net-ssh, as the inability to modify a hash during iteration is a Ruby limitation. See e.g: irb(main):005:0> h = { :a => 1} => {:a=>1} irb(main):006:0> h.each_pair { |k, v| h[:b] = 2 } RuntimeError: can't add a new key into hash during iteration from (irb):6:in `block in irb_binding' from (irb):6:in `each_pair' from (irb):6 from /usr/bin/irb:11:in `<main>' I'm closing this bug. Lucas
--- End Message ---
