[RE]Re: Kallithea 0.7.0 Running Environment Questions
Hello Kallithea, Thank you so much.We ended up upgradingthe EC2 instane from
t2.micro to r5.xlarge.So the specifications of the instance are as
follows.(Intel Xeon3.1GHzSkylake-SP orCascade Lake) vCPU = 4, RAM = 32GiBWe set
the thread value to 1 when you configure the ini file, the download will
proceed as a result, although it is slow.The thing is the server would not be
able to process anything else until the download is complete, so I want to
increase the thread value more in the ini file.I'd like to know the possible
risks.There is no other program running in the instance except for
Kallithea.- Original Message -From : Mads Kiilerich
To : "김태호" ,
Cc : "박정환" Sent
: 2021-06-17 06:07:21Subject : Re: Kallithea 0.7.0 Running Environment Questions
On 6/15/21 1:02 PM, 김태호 wrote:
We are testing by
installing 0.7.0 version of Kallithea in two different
environments.
One was installed on
WSL2 on my Windows 10 computer, and the other on EC2
(t2.Micro, Ubuntu20.04) on AWS.
The Kallithea git repo
that I want to download is about 2.84GB.
There was no problem when I installed it on my PC to verify
that the installation process or configuration was wrong.
If I run the Kallithea
in WSL2, there is no problem with the download.
The specifications of
WSL2 are as follows.
CPU : Intel(R) Core(TM)
i7-1065G7 CPU @ 1.30GHz 1.50 GHz ( 8 core )
RAM: 16 GB
EC2 is (t2.Micro)
vCPU:
1
RAM:
1
However, if I run on an
EC2 instance, the following error :
-->
start
error: RPC failed; HTTP
417 curl 22 The requested URL returned error: 417
fatal: the remote
end hung up unexpectedly
-->
end
Debug
level log txt fileis attached.
You saw this at the end of the log file:
2021-06-15 10:31:38.805 DEBUG
[kallithea.config.middleware.pygrack] handling cmd ['git',
'upload-pack', '--stateless-rpc',
'/var/kallithea/repos/Hicare-Smart/v2/hub-android']
2021-06-15 10:33:12.303 ERROR
[kallithea.config.middleware.pygrack] Traceback (most recent call
last):
File
"/home/ubuntu/kallithea/lib/python3.8/site-packages/kallithea/config/middleware/pygrack.py",
line 160, in backend
out = subprocessio.SubprocessIOChunker(
File
"/home/ubuntu/kallithea/lib/python3.8/site-packages/kallithea/lib/vcs/subprocessio.py",
line 365, in __init__
raise EnvironmentError("Subprocess exited due to an error: %s"
% err)
OSError: Subprocess exited due to an error: b'error: pack-objects
died of signal 9\nerror: git upload-pack: git-pack-objects died
with error.\nfatal: git upload-pack: aborting due to possible
repository corruption on the remote side.\n'
Kallithea is invoking 'git', and Git fails, probably because the
server is out of memory.
You can perhaps reproduce pretty much the same problem by running
this on the server:
cd /var/kallithea/repos/Hicare-Smart/v2/hub-android
git bundle create /tmp/bundle --all
On the machine where the operation works on the same repo, you
can try to use
/usr/bin/time -v git bundle create /tmp/bundle --all
and the line with "Maximum resident set size (kbytes)" will tell how
much memory it is using.
While it is hard to give any advice on server size, it seems
reasonable that the the server at least must be of similar size as
the repo, multiplied by some factor. Next, the server size will
depend on for example how many simultaneous operations it should
handle.
I guess it would work (but be slow) if the system is configured
with plenty of swap space. But real RAM is better.
/Mads
___
kallithea-general mailing list
kallithea-general@sfconservancy.org
https://lists.sfconservancy.org/mailman/listinfo/kallithea-general
Re: Kallithea 0.7.0 Running Environment Questions
On 6/15/21 1:02 PM, 김태호 wrote:
We are testing by installing 0.7.0 version of Kallithea in two
different environments.
One was installed on WSL2 on my Windows 10 computer, and the other on
EC2 (t2.Micro, Ubuntu20.04) on AWS.
The Kallithea git repo that I want to download is about 2.84GB.
There was no problem when I installed it on my PC to verify that the
installation process or configuration was wrong.
If I run the Kallithea in WSL2, there is no problem with the download.
The specifications of WSL2 are as follows.
CPU : Intel(R) Core(TM) i7-1065G7 CPU @ 1.30GHz 1.50 GHz ( 8 core )
RAM: 16 GB
EC2 is (t2.Micro)
vCPU: 1
RAM: 1
However, if I run on an EC2 instance, the following error :
--> start
error: RPC failed; HTTP 417 curl 22 The requested URL returned error: 417
fatal: the remote end hung up unexpectedly
--> end
Debug level log txt file is attached.
You saw this at the end of the log file:
2021-06-15 10:31:38.805 DEBUG [kallithea.config.middleware.pygrack]
handling cmd ['git', 'upload-pack', '--stateless-rpc',
'/var/kallithea/repos/Hicare-Smart/v2/hub-android']
2021-06-15 10:33:12.303 ERROR [kallithea.config.middleware.pygrack]
Traceback (most recent call last):
File
"/home/ubuntu/kallithea/lib/python3.8/site-packages/kallithea/config/middleware/pygrack.py",
line 160, in backend
out = subprocessio.SubprocessIOChunker(
File
"/home/ubuntu/kallithea/lib/python3.8/site-packages/kallithea/lib/vcs/subprocessio.py",
line 365, in __init__
raise EnvironmentError("Subprocess exited due to an error: %s" % err)
OSError: Subprocess exited due to an error: b'error: pack-objects died
of signal 9\nerror: git upload-pack: git-pack-objects died with
error.\nfatal: git upload-pack: aborting due to possible repository
corruption on the remote side.\n'
Kallithea is invoking 'git', and Git fails, probably because the server
is out of memory.
You can perhaps reproduce pretty much the same problem by running this
on the server:
cd /var/kallithea/repos/Hicare-Smart/v2/hub-android
git bundle create /tmp/bundle --all
On the machine where the operation works on the same repo, you can try
to use
/usr/bin/time -v git bundle create /tmp/bundle --all
and the line with "Maximum resident set size (kbytes)" will tell how
much memory it is using.
While it is hard to give any advice on server size, it seems reasonable
that the the server at least must be of similar size as the repo,
multiplied by some factor. Next, the server size will depend on for
example how many simultaneous operations it should handle.
I guess it would work (but be slow) if the system is configured with
plenty of swap space. But real RAM is better.
/Mads
___
kallithea-general mailing list
kallithea-general@sfconservancy.org
https://lists.sfconservancy.org/mailman/listinfo/kallithea-general
[RE]Re: Authentication log Questions
On 6/11/21 12:07 PM, 김태호 wrote: Hello Kallithea Thank you for your answer ! We have solved Question1 and Question3 but the second one. To help others with similar problems (and perhaps help us improve the software to avoid it), can you explain what the problem was and how you solved it? So, let me elaborate on the second question. The following attempts have been made to download kallithea-git repository which is about 150MB size: 1. Changed the http_server option from gearbox to waitress. This makes it possible to download more files than the gearbox, but the 150MB file still fails. *>> Command that I tired.* ubuntu@DESKTOP-VD6IGV2:~$ git clone https://taeho...@scm.dev.my-company.net/my-company-smart/v2/hub-android Cloning into 'hub-android'... Password for 'https://taeho...@scm.dev.my-company.net: error: RPC failed; HTTP 504 curl 22 The requested URL returned error: 504 fatal: the remote end hung up unexpectedly I will repeat the question from https://lists.sfconservancy.org/pipermail/kallithea-general/2021q2/003321.html : Do you get the same failure if bypassing the load balancer and connecting directly to the gearbox? I also asked: What do you get in the Kallithea logs for such requests - especially after setting the log levels to DEBUG? I verified that I can serve a clone of https://github.com/git/git/ with gearbox, and cloning that repo locally works: 2021-06-16 15:43:38.416 INFO [kallithea.config.middleware.wrapper] Request from 127.0.0.1 for /baregit/git-upload-pack responded '200 OK' after 11.572s with 163736351 bytes 4. Changed repository setting I changed *enable downloads *option. (checked the box) So I could download the zip file. I don't think I can keep using this because all my coworkers using Sourcetree app. But But I think it's going to be a clue to solve this problem. "Downloads" is for downloading a snapshot without version control. That is probably not a solution to your use case. /Mads ___ kallithea-general mailing list kallithea-general@sfconservancy.org https://lists.sfconservancy.org/mailman/listinfo/kallithea-general
