Hi Imesh,
I have almost completed the work to switch to gitpython. However there is
an issue with the type of the auth url, that is given to GitPython. Other
than that, public repo cloning now works with the library.
When a url with credentials is passed, I'm getting the following error.
Investigating further.
[2015-02-13 09:59:13,200] ERROR {eventsubscriber.py:run} - Error processing
'ArtifactUpdatedEvent' event
Traceback (most recent call last):
File
"/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/subscriber/eventsubscriber.py",
line 103, in run
handler(event_msg)
File
"/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/test-classes/../10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/agent.py",
line 185, in on_artifact_updated
CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
File
"/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/extensions/extensionhandler.py",
line 122, in on_artifact_updated_event
subscribe_run, updated =
agentgithandler.AgentGitHandler.checkout(repo_info)
File
"/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/artifactmgt/git/agentgithandler.py",
line 89, in checkout
git_repo = AgentGitHandler.clone(git_repo)
File
"/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/artifactmgt/git/agentgithandler.py",
line 192, in clone
Repo.clone_from(unicode(git_repo.repo_url, "utf-8"),
unicode(git_repo.local_repo_path, "utf-8"))
File "/usr/local/lib/python2.7/dist-packages/git/repo/base.py", line 844,
in clone_from
return cls._clone(Git(os.getcwd()), url, to_path, GitCmdObjectDB,
progress, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/git/repo/base.py", line 791,
in _clone
v=True, **add_progress(kwargs, git, progress))
File "/usr/local/lib/python2.7/dist-packages/git/cmd.py", line 431, in
<lambda>
return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/git/cmd.py", line 802, in
_call_process
return self.execute(make_call(), **_kwargs)
File "/usr/local/lib/python2.7/dist-packages/git/cmd.py", line 556, in
execute
**subprocess_kwargs
File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
TypeError: execve() arg 2 must contain only strings
Regards,
Chamila de Alwis
Software Engineer | WSO2 | +94772207163
Blog: code.chamiladealwis.com
On Fri, Feb 13, 2015 at 9:30 AM, Imesh Gunaratne <[email protected]> wrote:
> I analyzed the issue the NoneType error has occurred due to the EOF
> exception which has been raised while executing git clone shell command:
>
> The problem is in the following line:
>
> git_repo = AgentGitHandler.clone(git_repo)
> AgentGitHandler.log.debug("Git clone executed: [tenant-id] %s [repo-url] %s",
> git_repo.tenant_id, git_repo.repo_url)
>
> Here clone() method returns NoneType if clone fails. Anyway the root cause of
> the problem is with the Git client implementation. Will switch to GitPython
> to avoid this intermittent issue of EOF.
>
> Thanks
>
>
> On Fri, Feb 13, 2015 at 9:20 AM, Imesh Gunaratne <[email protected]> wrote:
>
>> I can see the tenant id attribute in received message, may be it was not
>> properly set:
>>
>> [2015-02-13 03:44:06,394] DEBUG {eventsubscriber.py:on_message} - Message
>> received: instance/notifier/ArtifactUpdatedEvent:
>>
>> {"clusterId":"php1.php.domain","repoUserName":"","repoPassword":"","repoURL":"
>> https://github.com/imesh/stratos-php-applications.git
>> ","tenantId":"-1234","commitEnabled":false}
>>
>> On Fri, Feb 13, 2015 at 9:19 AM, Imesh Gunaratne <[email protected]>
>> wrote:
>>
>>> Also the following error was there in the log:
>>>
>>> [2015-02-13 03:44:28,498] ERROR {agentgithandler.py:clone} - Exception
>>> while executing git clone command on git binary : EOF("End Of File (EOF).
>>> Exception style platform.\n<pexpect.spawn object at 0x25c17d0>\nversion:
>>> 3.3\ncommand: /usr/bin/git\nargs: ['/usr/bin/git', 'clone', '
>>> https://github.com/imesh/stratos-php-applications.git',
>>> '/var/www']\nsearcher: <pexpect.searcher_re object at 0x25c18d0>\nbuffer
>>> (last 100 chars): ''\nbefore (last 100 chars): 'ng deltas: 98% (71/72)
>>> \\rResolving deltas: 100% (72/72) \\rResolving deltas: 100% (72/72),
>>> done.\\r\\n'\nafter: <class 'pexpect.EOF'>\nmatch: None\nmatch_index:
>>> None\nexitstatus: None\nflag_eof: True\npid: 63\nchild_fd: 11\nclosed:
>>> False\ntimeout: 120\ndelimiter: <class 'pexpect.EOF'>\nlogfile:
>>> None\nlogfile_read: None\nlogfile_send: None\nmaxread: 2000\nignorecase:
>>> False\nsearchwindowsize: None\ndelaybeforesend: 0.05\ndelayafterclose:
>>> 0.1\ndelayafterterminate: 0.1",)
>>> Traceback (most recent call last):
>>> File
>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>> line 200, in clone
>>> result = clone_op.expect(["Username for .*", "Password for .*",
>>> "Checking connectivity... done."])
>>> File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py",
>>> line 1451, in expect
>>> timeout, searchwindowsize)
>>> File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py",
>>> line 1466, in expect_list
>>> timeout, searchwindowsize)
>>> File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py",
>>> line 1554, in expect_loop
>>> raise EOF(str(err) + '\n' + str(self))
>>> EOF: End Of File (EOF). Exception style platform.
>>> <pexpect.spawn object at 0x25c17d0>
>>>
>>> On Fri, Feb 13, 2015 at 9:17 AM, Imesh Gunaratne <[email protected]>
>>> wrote:
>>>
>>>> The problem was in populate-user-data.sh file [1], APPLICATION_PATH has
>>>> been removed from it. I just fixed it, pushed the new PHP docker image to
>>>> docker hub and did another flow test and found the below error:
>>>>
>>>> [2015-02-13 03:44:28,603] ERROR {eventsubscriber.py:run} - Error
>>>> processing 'ArtifactUpdatedEvent' event
>>>> Traceback (most recent call last):
>>>> File
>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>> line 103, in run
>>>> handler(event_msg)
>>>> File "agent.py", line 185, in on_artifact_updated
>>>>
>>>> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
>>>> File
>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py",
>>>> line 122, in on_artifact_updated_event
>>>> subscribe_run, updated =
>>>> agentgithandler.AgentGitHandler.checkout(repo_info)
>>>> File
>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>> line 91, in checkout
>>>> git_repo.tenant_id, git_repo.repo_url)
>>>> AttributeError: 'NoneType' object has no attribute 'tenant_id'
>>>>
>>>> [1]
>>>> https://github.com/apache/stratos/blob/master/tools/docker-images/cartridge-docker-images/base-image/files/populate-user-data.sh
>>>>
>>>> On Fri, Feb 13, 2015 at 7:43 AM, Imesh Gunaratne <[email protected]>
>>>> wrote:
>>>>
>>>>> Thanks Lakmal, yes as I found basic features we need works with Git
>>>>> python. Will do that.
>>>>> Regarding the above error, seems like it has occurred due to the
>>>>> APPLICATION_PATH not being sent in payload, there was no None check or a
>>>>> log saying its not found:
>>>>>
>>>>> def clone(git_repo):
>>>>> if os.path.isdir(git_repo.local_repo_path):
>>>>> # delete and recreate local repo path if exists
>>>>> GitUtils.delete_folder_tree(git_repo.local_repo_path)
>>>>>
>>>>>
>>>>> On Fri, Feb 13, 2015 at 7:37 AM, Lakmal Warusawithana <[email protected]
>>>>> > wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Feb 13, 2015 at 7:34 AM, Imesh Gunaratne <[email protected]>
>>>>>> wrote:
>>>>>>
>>>>>>> As it looks like this error has been caused by the agent git handler
>>>>>>> while executing git clone, IMO its better if we can switch to GitPython.
>>>>>>>
>>>>>>>
>>>>>> +1, If GitPython has full functionality what we expect, lets move to
>>>>>> it. I was under impression that it has some limitation. If so please not
>>>>>> waist time to fix current one.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>> On Fri, Feb 13, 2015 at 7:31 AM, Imesh Gunaratne <[email protected]>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Devs,
>>>>>>>>
>>>>>>>> Yesterday we fixed several issues in PCA, now I'm seeing the below
>>>>>>>> error when trying to receive artifact updated event:
>>>>>>>>
>>>>>>>> [2015-02-13 01:56:07,311] ERROR {eventsubscriber.py:run} - Error
>>>>>>>> processing 'ArtifactUpdatedEvent' event
>>>>>>>> Traceback (most recent call last):
>>>>>>>> File
>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>>>>>> line 103, in run
>>>>>>>> handler(event_msg)
>>>>>>>> File "agent.py", line 185, in on_artifact_updated
>>>>>>>>
>>>>>>>> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
>>>>>>>> File
>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py",
>>>>>>>> line 122, in on_artifact_updated_event
>>>>>>>> subscribe_run, updated =
>>>>>>>> agentgithandler.AgentGitHandler.checkout(repo_info)
>>>>>>>> File
>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>>>>>> line 89, in checkout
>>>>>>>> git_repo = AgentGitHandler.clone(git_repo)
>>>>>>>> File
>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>>>>>> line 190, in clone
>>>>>>>> if os.path.isdir(git_repo.local_repo_path):
>>>>>>>> File "/usr/lib/python2.7/genericpath.py", line 41, in isdir
>>>>>>>> st = os.stat(s)
>>>>>>>> TypeError: coercing to Unicode: need string or buffer, NoneType foun
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Imesh Gunaratne
>>>>>>>>
>>>>>>>> Technical Lead, WSO2
>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Imesh Gunaratne
>>>>>>>
>>>>>>> Technical Lead, WSO2
>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Lakmal Warusawithana
>>>>>> Vice President, Apache Stratos
>>>>>> Director - Cloud Architecture; WSO2 Inc.
>>>>>> Mobile : +94714289692
>>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Imesh Gunaratne
>>>>>
>>>>> Technical Lead, WSO2
>>>>> Committer & PMC Member, Apache Stratos
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Imesh Gunaratne
>>>>
>>>> Technical Lead, WSO2
>>>> Committer & PMC Member, Apache Stratos
>>>>
>>>
>>>
>>>
>>> --
>>> Imesh Gunaratne
>>>
>>> Technical Lead, WSO2
>>> Committer & PMC Member, Apache Stratos
>>>
>>
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>
>
>
> --
> Imesh Gunaratne
>
> Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>
