Behavior of create_repo API as a general user

[toras]

Hi

I have doubts about the behavior of 'create_repo' in Kallithea's API, so 
I will post it.

The version of kallithea I'm using is 0.6.3.

A 'create_repo' request to a repository group for which the account has 
write permissions also appears to fail if top-level repository creation 
is disabled.
The same request succeeds when I enable the create top-level repository 
setting.
Regardless of top-level settings, I can use that account to create 
repositories from the web into repository groups.


I didn't understand if the explanation of 'Note' on the setting screen 
means "Failed even if I have write permission".


For the time being, the situation I tried is described below.

The request was made like this.
```
curl http://localhost:5000/_admin/api -X POST -H 
'content-type:text/plain' --data-binary 
'{"id":1,"api_key":"0ae8322ce787f08771c6b3570765318fb0360ad6","method":"create_repo","args":{"repo_name":"grp/test", 
"repo_type":"git"}}'

```

The response in case of failure is like this.
```
{"id": 1, "result": null, "error": "Internal server error"}
```

The console output of kallithea at that time looks like the following.
```
2021-01-02 17:25:23.087 DEBUG [JSONRPC] Trying to find JSON-RPC method: 
create_repo
2021-01-02 17:25:23.087 INFO  [JSONRPC] IP: 127.0.0.1 Request to 
/_admin/api time: 0.012s
2021-01-02 17:25:23.127 ERROR [JSONRPC] Encountered unhandled exception: 
Traceback (most recent call last):
  File 
"/home/kallithea/.local/lib/python3.6/site-packages/kallithea/controllers/api/__init__.py", 
line 225, in _rpc_call

raw_response = getattr(self, action)(**rpc_args)
  File "", line 2, in create_repo
  File 
"/home/kallithea/.local/lib/python3.6/site-packages/kallithea/lib/auth.py", 
line 664, in __wrapper

raise HTTPForbidden()
webob.exc.HTTPForbidden: Access was denied to this resource.
```

# I rely on translation tools. I'm sorry if there is a strange sentence.


Thanks


toras9000

___
kallithea-general mailing list
kallithea-general@sfconservancy.org
https://lists.sfconservancy.org/mailman/listinfo/kallithea-general

Re: Behavior of create_repo API as a general user

[toras]

Hi

Thank you for your answer.

I understand that it is a process of changing the interpretation of this 
setting value.

I'm looking forward to future versions including behavior fixes.

Thanks


toras9000

On 2021/01/03 7:49, Mads Kiilerich wrote:

Hi

You are right. Kallithea has some bugs around API permission handling. It is not
using the "create top-level repositories" permissions correctly.

This problem is related to the
"This will also give all users API access to create repositories everywhere.
That might change in future versions."
note, even though you see the opposite problem.

This behaviour is kind of intentional -
https://kallithea-scm.org/repos/kallithea/changeset/6620542597d3 - and with some
awareness in the test suite -
https://kallithea-scm.org/repos/kallithea-incoming/changeset/975f5769be08 ...
but doesn't match what hg.create.repositoryactually means:
https://kallithea-scm.org/repos/kallithea/changeset/8aad6a324739#kallitheamodeldbpy_n1676

I propose
https://kallithea-scm.org/repos/kallithea/pull-request/303/_/api_permission_check
 to
fix this.

/Mads



___
kallithea-general mailing list
kallithea-general@sfconservancy.org
https://lists.sfconservancy.org/mailman/listinfo/kallithea-general

Re: Behavior of create_repo API as a general user

[toras]

Thank you for the feature fixes and notifications.
I will try it after the next version is released.

Thanks


toras9000

On 2021/01/16 6:25, Mads Kiilerich wrote:
This has now been fixed in the stable branch and will be included in the 
next release - no matter if it will be 0.6.4 or 0.7 .


Thanks for the report.

/Mads



___
kallithea-general mailing list
kallithea-general@sfconservancy.org
https://lists.sfconservancy.org/mailman/listinfo/kallithea-general

Re: Requirements error in Kallithea 0.7.0

[toras]

Hello

I'm not familiar with this issue, but I'm posting this for your 
information.
I encountered a similar problem with docker ubuntu image (Install pip 
with get-pip.py).


In the error log,
"Unknown distribution option: 'convert_2to3_doctests'"
and
"error in FormEncode setup command: use_2to3 is invalid."
message is included.

Maybe this has something to do with it.
https://github.com/pypa/setuptools/issues/1120

Apparently, it means that 2to3 is no longer available in setuptools 58?



On 2021/09/15 18:23, Thomas De Schampheleire wrote:

Hello,

El mié, 15 sept 2021 a las 10:57, Tom Gelsthorpe
() escribió:


Hi,

please have a look at this, it was working until about 10 days ago

   pip install --upgrade kallithea==0.7.0

but now I get

ERROR: Could not find a version that satisfies the requirement
FormEncode<1.4,>=1.3.1 (from kallithea) (from versions: 0.2, 0.2.1,
0.2.2, 0.3, 0.4, 0.5, 0.5.1, 0.6, 0.7, 0.7.1, 0.9, 1.0, 1.0.1, 1.1, 1.2,
1.2.1, 1.2.2, 1.2.3.dev0, 1.2.4, 1.2.5, 1.2.6, 1.3.0a1, 1.3.0, 1.3.1,
2.0.0a1, 2.0.0)
ERROR: No matching distribution found for FormEncode<1.4,>=1.3.1

so FormEncode went from 1.3.1 to 2.0.0

the version check in setup.py at line 53 fails:

"FormEncode >= 1.3.1, < 1.4",



I don't see this problem.
I created a fresh virtualenv and ran the command you showed.
I get following set of packages:

pip freeze
alembic==1.4.3
amqp==5.0.6
Babel==2.8.1
backlash==0.3.1
bcrypt==3.1.7
Beaker==1.11.0
billiard==3.6.4.0
bleach==3.1.3
celery==5.0.5
certifi==2021.5.30
cffi==1.14.6
chardet==4.0.0
click==7.1.2
click-didyoumean==0.0.3
click-plugins==1.1.1
click-repl==0.2.0
crank==0.8.1
decorator==4.4.2
docutils==0.16
dulwich==0.19.16
FormEncode==1.3.1
gearbox==0.2.1
hupper==1.10.3
ipaddr==2.2.0
Kallithea==0.7.0
kombu==5.1.0
Mako==1.1.5
Markdown==3.1.1
MarkupSafe==2.0.1
mercurial==5.8.1
paginate==0.5.6
paginate-sqlalchemy==0.3.1
Paste==3.4.6
PasteDeploy==2.1.1
prompt-toolkit==3.0.20
pycparser==2.20
Pygments==2.6.1
python-dateutil==2.8.2
python-editor==1.0.4
pytz==2021.1
repoze.lru==0.7
Routes==2.4.1
six==1.16.0
SQLAlchemy==1.3.24
Tempita==0.5.2
tgext.routes==0.2.1
TurboGears2==2.4.3
urllib3==1.26.6
URLObject==2.4.3
vine==5.0.0
waitress==1.4.4
wcwidth==0.2.5
webencodings==0.5.1
WebHelpers2==2.0
WebOb==1.8.7
Whoosh==2.7.4



Note that in the list in your output, FormEncode 1.3.1 is still
present and is the intended version.

It looks to me as if there is another package in your virtualenv that
explicitly expects another version.
Do you get the same problem from a fresh virtualenv?

I think it must be possible to let pip give more verbose output about
why it did not consider FormEncode 1.3.1.

Best regards,
Thomas
___
kallithea-general mailing list
kallithea-general@sfconservancy.org
https://lists.sfconservancy.org/mailman/listinfo/kallithea-general



___
kallithea-general mailing list
kallithea-general@sfconservancy.org
https://lists.sfconservancy.org/mailman/listinfo/kallithea-general

Raw download and filename character code

[toras]

Hello.

In Kallithea 0.7.0, there was a problem when raw downloading the file in the 
repository from the web interface.
It seems to be an error if the filename registered in the repository cannot be 
encoded with 'latin-1'.
It's not a big issue, as you can work around the problem by cloning the 
repository, but I'll report it here.

When an error occurs, the following will be displayed on the browser.

Internal Server Error
The server encountered an unexpected internal server error
(generated by waitress)


In addition, the following traceback is output to the console.

Traceback (most recent call last):
  File 
"/home/kallithea/.local/lib/python3.8/site-packages/waitress/channel.py", line 
350, in service
task.service()
  File "/home/kallithea/.local/lib/python3.8/site-packages/waitress/task.py", 
line 171, in service
self.execute()
  File "/home/kallithea/.local/lib/python3.8/site-packages/waitress/task.py", 
line 479, in execute
self.write(chunk)
  File "/home/kallithea/.local/lib/python3.8/site-packages/waitress/task.py", 
line 311, in write
rh = self.build_response_header()
  File "/home/kallithea/.local/lib/python3.8/site-packages/waitress/task.py", 
line 284, in build_response_header
return tobytes(res)
  File "/home/kallithea/.local/lib/python3.8/site-packages/waitress/compat.py", 
line 69, in tobytes
return bytes(s, "latin-1")
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 84-85: 
ordinal not in range(256)


Thanks

--
toras9000


___
kallithea-general mailing list
kallithea-general@sfconservancy.org
https://lists.sfconservancy.org/mailman/listinfo/kallithea-general

Re: Raw download and filename character code

[toras]

Hello.

I'm sorry, I should have given a concrete example.

Thank you for creating the fix code.
From the conclusion, I've confirmed that the fixed code downloaded the file 
correctly.

I tried overwriting the fixed file for kallithea installed by pip.
The repository type I tried is git, and the stored file is a name that contains 
Japanese characters.
# I don't know if this can be displayed correctly in the email, but the file name is 
"メモ.txt".


Thanks

--
toras9000

On 2022/02/08 8:25, Mads Kiilerich wrote:

Hi

Thanks for the report.

It took a while to reproduce. Many filenames work ok. An exact example would 
have been helpful.

I reproduced the problem (or another similar one) on Linux with a Mercurial 
repository with a file created as
   python -c 'open("\U0001f4a9", "w").write("\U0001f4a9")'

I pushed a fix to the stable branch on 
https://kallithea-scm.org/repos/kallithea/changeset/8993d401575bad2d085158738b09e0bd02c7f135 . Please let me know if that 
doesn't solve your problem.


/Mads





___
kallithea-general mailing list
kallithea-general@sfconservancy.org
https://lists.sfconservancy.org/mailman/listinfo/kallithea-general

Edit user group permissions

[toras]

Hello.


In kallithea v0.7.0.
When I press the [Add new] link on the user group permissions setting page, it immediately displays `User group permissions 
updated`.

Looking at the browser's DevTool, it appears to be making a POST request to the 
same page and getting HTTP status 302.

Is this a problem and isn't working?
Or is the functionality not yet implemented?


Thanks

--
toras9000


___
kallithea-general mailing list
kallithea-general@sfconservancy.org
https://lists.sfconservancy.org/mailman/listinfo/kallithea-general

Re: Edit user group permissions

[toras]

Hi


Thank you for introducing the details of fix.
I will apply that fix and use it.
Very helpful.


Thanks

On 2022/06/28 4:17, Mads Kiilerich wrote:

Hi

Right. I reproduced on http://localhost:5000/_admin/user_groups/1/edit/perms 
and fixed it (and other similar potential problems).

See 
https://kallithea-scm.org/repos/kallithea/changeset/cc70797e70e16163c6862a42f7f045fb6782f202
 .

For now, you can upgrade to the head of the stable branch, or you can apply the 
fix manually.

/Mads



On 6/27/22 17:34, toras wrote:

Hello.


In kallithea v0.7.0.
When I press the [Add new] link on the user group permissions setting page, it immediately displays `User group permissions 
updated`.

Looking at the browser's DevTool, it appears to be making a POST request to the 
same page and getting HTTP status 302.

Is this a problem and isn't working?
Or is the functionality not yet implemented?


Thanks

--
toras9000


___
kallithea-general mailing list
kallithea-general@sfconservancy.org
https://lists.sfconservancy.org/mailman/listinfo/kallithea-general





___
kallithea-general mailing list
kallithea-general@sfconservancy.org
https://lists.sfconservancy.org/mailman/listinfo/kallithea-general

Error in config value 'url_scheme_variable'

[toras]

Hello.


I am reporting a problem I encountered when trying to deploy kallithhea behind 
a reverse proxy.
Version is v0.7.0.

Enabling the setting 'url_scheme_variable = HTTP_X_FORWARDED_PROTO' in the configuration file (kallithea.ini) caused an error on 
startup.

A portion of the stack trace output when an error occurs is shown below.


2022-09-12 19:38:41.258 ERROR [gearbox] String is not true/false: 
'http_x_forwarded_proto'
Traceback (most recent call last):
<< Omit the middle >>
  File 
"/home/kallithea/.local/lib/python3.8/site-packages/kallithea/config/application.py", 
line 38, in 
if any(asbool(config.get(x)) for x in ['url_scheme_variable', 
'force_https', 'use_htsts']):


Thanks

--
toras9000


___
kallithea-general mailing list
kallithea-general@sfconservancy.org
https://lists.sfconservancy.org/mailman/listinfo/kallithea-general

Re: Error in config value 'url_scheme_variable'

[toras]

Hi

Thank you for confirming and addressing this.

--
toras9000

On 2022/09/14 16:10, Mads Kiilerich wrote:

Hi

Right. Thanks for reporting.

That should be fixed with 
https://kallithea-scm.org/repos/kallithea/changeset/ed117efc9ae952bbab966a267bbd2297d31b05e2
 .

You can apply the fix manually, or install from the stable branch - 
https://kallithea.readthedocs.io/en/default/installation.html#installation-from-repository-source .


/Mads


On 9/13/22 17:29, toras wrote:

Hello.


I am reporting a problem I encountered when trying to deploy kallithhea behind 
a reverse proxy.
Version is v0.7.0.

Enabling the setting 'url_scheme_variable = HTTP_X_FORWARDED_PROTO' in the configuration file (kallithea.ini) caused an error 
on startup.

A portion of the stack trace output when an error occurs is shown below.


2022-09-12 19:38:41.258 ERROR [gearbox] String is not true/false: 
'http_x_forwarded_proto'
Traceback (most recent call last):
<< Omit the middle >>
  File 
"/home/kallithea/.local/lib/python3.8/site-packages/kallithea/config/application.py", 
line 38, in 
    if any(asbool(config.get(x)) for x in ['url_scheme_variable', 
'force_https', 'use_htsts']):


Thanks

--
toras9000


___
kallithea-general mailing list
kallithea-general@sfconservancy.org
https://lists.sfconservancy.org/mailman/listinfo/kallithea-general





___
kallithea-general mailing list
kallithea-general@sfconservancy.org
https://lists.sfconservancy.org/mailman/listinfo/kallithea-general

About API code comments and some behavior.

[toras]

Hi.


There was an API-related topic a little while ago,so there may be things that are currently being changed, but I recently 
noticed something while writing a client that uses the API, so I'd like to report it.


- The comments in api.py have some differences from the implementation.
  I tried calling the API and attached a file that changed the part that is 
different from the result.
  (Based on commit 7037365a7bc3.)
  I don't know if this is necessary, but for your information.

- The 'parent' parameter of 'update_repo_group' does not work.
  It appears to accept the 'parent' parameter, but specifying it seems to have 
no effect.


Thanks

--
toras9000

# -*- coding: utf-8 -*-
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program.  If not, see .
"""
kallithea.controllers.api.api
~

API controller for Kallithea

This file was forked by the Kallithea project in July 2014.
Original author and date, and relevant copyright and licensing information is 
below:
:created_on: Aug 20, 2011
:author: marcink
:copyright: (c) 2013 RhodeCode GmbH, and others.
:license: GPLv3, see LICENSE.md for more details.
"""

import logging
import traceback
from datetime import datetime

from tg import request

from kallithea.controllers.api import JSONRPCController, JSONRPCError
from kallithea.lib.auth import (AuthUser, HasPermissionAny, 
HasPermissionAnyDecorator, HasRepoGroupPermissionLevel, HasRepoPermissionLevel,
HasUserGroupPermissionLevel)
from kallithea.lib.exceptions import DefaultUserException, 
UserGroupsAssignedException
from kallithea.lib.utils import repo2db_mapper
from kallithea.lib.vcs.backends.base import EmptyChangeset
from kallithea.lib.vcs.exceptions import EmptyRepositoryError
from kallithea.model import db, meta, userlog
from kallithea.model.changeset_status import ChangesetStatusModel
from kallithea.model.comment import ChangesetCommentsModel
from kallithea.model.gist import GistModel
from kallithea.model.pull_request import PullRequestModel
from kallithea.model.repo import RepoModel
from kallithea.model.repo_group import RepoGroupModel
from kallithea.model.scm import ScmModel, UserGroupList
from kallithea.model.user import UserModel
from kallithea.model.user_group import UserGroupModel


log = logging.getLogger(__name__)


def store_update(updates, attr, name):
"""
Stores param in updates dict if it's not None (i.e. if user explicitly set
a parameter). This allows easy updates of passed in params.
"""
if attr is not None:
updates[name] = attr


def get_user_or_error(userid):
"""
Get user by id or name or return JsonRPCError if not found

:param userid:
"""
user = UserModel().get_user(userid)
if user is None:
raise JSONRPCError("user `%s` does not exist" % (userid,))
return user


def get_repo_or_error(repoid):
"""
Get repo by id or name or return JsonRPCError if not found

:param repoid:
"""
repo = RepoModel().get_repo(repoid)
if repo is None:
raise JSONRPCError('repository `%s` does not exist' % (repoid,))
return repo


def get_repo_group_or_error(repogroupid):
"""
Get repo group by id or name or return JsonRPCError if not found

:param repogroupid:
"""
repo_group = db.RepoGroup.guess_instance(repogroupid)
if repo_group is None:
raise JSONRPCError(
'repository group `%s` does not exist' % (repogroupid,))
return repo_group


def get_user_group_or_error(usergroupid):
"""
Get user group by id or name or return JsonRPCError if not found

:param usergroupid:
"""
user_group = UserGroupModel().get_group(usergroupid)
if user_group is None:
raise JSONRPCError('user group `%s` does not exist' % (usergroupid,))
return user_group


def get_perm_or_error(permid, prefix=None):
"""
Get permission by id or name or return JsonRPCError if not found

:param permid:
"""
perm = db.Permission.get_by_key(permid)
if perm is None:
raise JSONRPCError('permission `%s` does not exist' % (permid,))
if prefix:
if not perm.permission_name.startswith(prefix):
raise JSONRPCError('permission `%s` is invalid, '
   'should start with %s' % (permid, prefix))
return perm


def get_gist_or_error(gistid):
"""
Get gist by id or gist_access_id or return Js

Re: About API code comments and some behavior.

[toras]

Hi


> (If you are working from a repo checkout, it would be more helpful if you 
could send a diff.)

You are right. I wasn't used to it.
I will do so on future occasions.

>> +"gist": 
> What is gist object? That should perhaps be clarified. Or perhaps it is a bug 
that it is returned ...

I wrote it in the sense that the  in the result of this 'create_gist' is JSON with the same structure as the result 
of 'get_gist'.

It's information about the created gist and I think it's the correct return 
value.
To avoid misinterpretation as an empty object, I imitated 'create_user' comment 
and wrote it like this.

Specifically,  was such JSON data.
  "gist": {"gist_id": 1, "type": "public", "access_id": "1", "description": "", "url": "http://localhost:/_admin/gists/1";, 
"expires": -1.0, "created_on": "2022-10-14T14:20:24.625"}



>> - The 'parent' parameter of 'update_repo_group' does not work.
> A quick look:  The update_repo_group API arguments seems to be handled by
> 
https://kallithea-scm.org/repos/kallithea/files/7037365a/kallithea/model/repo_group.py#L278
 . So perhaps the code in api.py
> should pass it as 'parent_group_id' instead of 'parent_group'?
> (But also, 'owner' doesn't seem to be handled at all. Does owner change 
really work for you? But also, I don't think it matters
> much who owns a repo group. Admin rights does the same thing. So repo group 
owner should perhaps just be removed from api.py and
> documentation?)

I'm not confident in my reading comprehension, but...

As you say, it looks like it would be necessary to pass it by 'parent_group_id'.
Also, is it necessary to not pass the 'parent_group_id' if it is not moved?
And I think that api.py should also resolve with get_repo_group_or_error() so 
that both name and id values are available.

And 'owner' certainly doesn't work either. I hadn't noticed.
Looking at the comments on RepoGroupModel.create(), did the implementer plan to 
use owner in the future?
There is also a countermeasure to make it changeable with update, right?
But for consistency in the current situation, it seems reasonable to remove it 
from the API parameters and documentation.


> Anyway:
> I also pushed some further cleanup of the documentation in api.py and api.rst 
.That's on the *stable* branch. It would be great
> if they could converge, and we could generate api.rst from api.py .
> If you want to improve documentation further, take a look at 
https://kallithea-scm.org/repos/kallithea/pull-request/325/_/api
> and propose api.py changes to make a grand unification.

Thank you. But as of now I do not have any further changes.



Thanks

--
toras9000

On 2022/10/14 21:51, Mads Kiilerich wrote:

Hi

Thank you. (If you are working from a repo checkout, it would be more helpful if you could send a diff.) I haven't verified in 
detail, but all the changes seem plausible ;-) I have pushed them to the stable branch.


One question though:


   id : 
   result : {
 "msg": "created new gist",
-    "gist": {}
+    "gist": 
   }
   error :  null


What is gist object? That should perhaps be clarified. Or perhaps it is a bug 
that it is returned ...


- The 'parent' parameter of 'update_repo_group' does not work.
  It appears to accept the 'parent' parameter, but specifying it seems to have no effect. 


A quick look:  The update_repo_group API arguments seems to be handled by 
https://kallithea-scm.org/repos/kallithea/files/7037365a/kallithea/model/repo_group.py#L278 . So perhaps the code in api.py 
should pass it as 'parent_group_id' instead of 'parent_group'?
(But also, 'owner' doesn't seem to be handled at all. Does owner change really work for you? But also, I don't think it matters 
much who owns a repo group. Admin rights does the same thing. So repo group owner should perhaps just be removed from api.py and 
documentation?)


Anyway:
I also pushed some further cleanup of the documentation in api.py and api.rst .That's on the *stable* branch. It would be great 
if they could converge, and we could generate api.rst from api.py .
If you want to improve documentation further, take a look at https://kallithea-scm.org/repos/kallithea/pull-request/325/_/api 
and propose api.py changes to make a grand unification.


/Mads


On 13/10/2022 15:00, toras wrote:

Hi.


There was an API-related topic a little while ago,so there may be things that are currently being changed, but I recently 
noticed something while writing a client that uses the API

Re: About API code comments and some behavior.

[toras]

Hi

I then noticed that the enable_downloads/enable_statistics parameters in the 
create_repo API also had no effect.
It seems that no parameters are used and the repository's default settings are used. (I think this is probably the expected 
behavior.)

I think this one also needs to be removed from the API parameters and 
description.


Thanks

--
toras9000

On 2022/10/15 12:31, toras wrote:

Hi


 > (If you are working from a repo checkout, it would be more helpful if you 
could send a diff.)

You are right. I wasn't used to it.
I will do so on future occasions.

 >> +    "gist": 
 > What is gist object? That should perhaps be clarified. Or perhaps it is a 
bug that it is returned ...

I wrote it in the sense that the  in the result of this 'create_gist' is JSON with the same structure as the result 
of 'get_gist'.

It's information about the created gist and I think it's the correct return 
value.
To avoid misinterpretation as an empty object, I imitated 'create_user' comment 
and wrote it like this.

Specifically,  was such JSON data.
   "gist": {"gist_id": 1, "type": "public", "access_id": "1", "description": "", "url": "http://localhost:/_admin/gists/1";, 
"expires": -1.0, "created_on": "2022-10-14T14:20:24.625"}



 >> - The 'parent' parameter of 'update_repo_group' does not work.
 > A quick look:  The update_repo_group API arguments seems to be handled by
 > 
https://kallithea-scm.org/repos/kallithea/files/7037365a/kallithea/model/repo_group.py#L278
 . So perhaps the code in api.py
 > should pass it as 'parent_group_id' instead of 'parent_group'?
 > (But also, 'owner' doesn't seem to be handled at all. Does owner change 
really work for you? But also, I don't think it matters
 > much who owns a repo group. Admin rights does the same thing. So repo group 
owner should perhaps just be removed from api.py and
 > documentation?)

I'm not confident in my reading comprehension, but...

As you say, it looks like it would be necessary to pass it by 'parent_group_id'.
Also, is it necessary to not pass the 'parent_group_id' if it is not moved?
And I think that api.py should also resolve with get_repo_group_or_error() so 
that both name and id values are available.

And 'owner' certainly doesn't work either. I hadn't noticed.
Looking at the comments on RepoGroupModel.create(), did the implementer plan to 
use owner in the future?
There is also a countermeasure to make it changeable with update, right?
But for consistency in the current situation, it seems reasonable to remove it 
from the API parameters and documentation.


 > Anyway:
 > I also pushed some further cleanup of the documentation in api.py and 
api.rst .That's on the *stable* branch. It would be great
 > if they could converge, and we could generate api.rst from api.py .
 > If you want to improve documentation further, take a look at 
https://kallithea-scm.org/repos/kallithea/pull-request/325/_/api
 > and propose api.py changes to make a grand unification.

Thank you. But as of now I do not have any further changes.



Thanks



___
kallithea-general mailing list
kallithea-general@sfconservancy.org
https://lists.sfconservancy.org/mailman/listinfo/kallithea-general

Re: About API code comments and some behavior.

[toras]

Hi

I still rely on translation tools.
Perhaps it is hard to read. I hope I can convey it properly.
I also tried attaching the patch, but I should add that I'm inexperienced with 
python.


> Yes. Can you confirm this will solve the code problems you reported:
> 
https://kallithea-scm.org/repos/kallithea-incoming/changeset/088551a24485247af328f0b8e395fdbbcd62a700
 ?

1. Modify create_repo()
  I don't think this change is correct.
  I overwrote api.py and called create_repo, but 
enable_downloads/enable_statistics seems to have no effect.

  I think the call flow during API execution is as follows.

  api.py : ApiController.create_repo()
repo.py : RepoModel.create()
  repo.py : (@celerylib.task) create_repo()

  I think that the third function does not refer to form_data, but refers to 
defs to obtain the default value and use it.
  
https://kallithea-scm.org/repos/kallithea/files/2dd317e9cc4b5ff94c1ddae7ee772d20b61259a9/kallithea/model/repo.py#L717
  I think this is probably implemented in conjunction with the web front end.

  I see two options for changes to the create_repo API.
  I think this requires a policy decision.

  Scenario 1:
The API will no longer accept that parameter.
Attachment: 1-scenario1.patch

  Scenario 2:
Support parameter specification in RepoModel.
Attachment: 1-scenario2.patch

2. Modify update_repo_group()
  I think there are two issues with parent handling.

  The first is.
  Other api parameters can basically target entities by name.
  It seems to me that as the parent_group_id parameter to RepoGroupModel.update(), the API parameter should be passed with name 
resolution instead of passing it as is.

Attachment: 2-parent_id.patch

  The second is still unresolved and not yet known in detail.
  Only information known at this time is provided.
  The problem is that changing the parent group via the API will leave the 
database and file system in an inconsistent state.
  At that time, new_path does not seem to indicate the correct path in 
RepoGroupModel.update(). So the folder is not moved.
  It seems correct when run from the web front end. I still don't know what the 
difference is.


Supplementation.
It may not be necessary, but I will publish the docker files and scripts that I 
used in the confirmation work.
https://github.com/toras9000/kallithea_api_test



Thanks
--
toras9000

On 2022/10/29 17:36, Mads Kiilerich wrote:

Hi

Yes. Can you confirm this will solve the code problems you reported: 
https://kallithea-scm.org/repos/kallithea-incoming/changeset/088551a24485247af328f0b8e395fdbbcd62a700 ?


/Mads


On 29/10/2022 09:28, toras wrote:

Hi

I then noticed that the enable_downloads/enable_statistics parameters in the 
create_repo API also had no effect.
It seems that no parameters are used and the repository's default settings are used. (I think this is probably the expected 
behavior.)

I think this one also needs to be removed from the API parameters and 
description.


Thanks



# HG changeset patch
# User toras9000
# Date 1667130294 -32400
#  Sun Oct 30 20:44:54 2022 +0900
# Branch create_repo_scenario1
# Node ID 6e77e0c3353c51e393b9fe70fc7ed4ce4887139c
# Parent  088551a24485247af328f0b8e395fdbbcd62a700
delete create_repo params

diff --git a/kallithea/controllers/api/api.py b/kallithea/controllers/api/api.py
--- a/kallithea/controllers/api/api.py
+++ b/kallithea/controllers/api/api.py
@@ -1133,8 +1133,6 @@
 repo_type=None, description='',
 private=False, clone_uri=None,
 landing_rev='rev:tip',
-enable_statistics=None,
-enable_downloads=None,
 copy_permissions=False):
 """
 Creates a repository. The repository name contains the full path, but 
the
@@ -1158,10 +1156,6 @@
 :type clone_uri: str
 :param landing_rev: :
 :type landing_rev: str
-:param enable_downloads:
-:type enable_downloads: bool
-:param enable_statistics:
-:type enable_statistics: bool
 :param copy_permissions: Copy permission from group that repository is
 being created.
 :type copy_permissions: bool
@@ -1215,10 +1209,6 @@
 private = defs.get('repo_private') or False
 if repo_type is None:
 repo_type = defs.get('repo_type')
-if enable_statistics is None:
-enable_statistics = defs.get('repo_enable_statistics')
-if enable_downloads is None:
-enable_downloads = defs.get('repo_enable_downloads')
 
 try:
 data = dict(
@@ -1230,8 +1220,6 @@
 clone_uri=clone_uri,
 repo_group=group_name,
 repo_landing_rev=landing_rev,
-repo_enable_statistics=enable_statistics,
-repo_enable_downloads=enable_down

Re: About API code comments and some behavior.

[toras]

Hi


I have found the cause of the problem with the update_repo_group API behavior 
that I wrote about in my previous email.
The implementation of RepoGroupModel.update() does not seem to work correctly unless it includes a valid group_name in args, 
even if the name is not changed.

If no new name is specified for the API parameter, it may be necessary to pass 
the original name.
I have created a patch to api.py based on 088551a24485 from kallithea-imcoming.
Attachment: fix-update_repo_group-1.patch


Also, I thought it was not good to have inconsistent results depending on the call parameters, so I refactored the 
implementation of RepoGroupModel.update().

It also includes changes that correct incorrect logging output. However, this 
is not required for API operation.
Attachment: fix-update_repo_group-2.patch


Thanks

On 2022/10/30 21:00, toras wrote:

Hi

I still rely on translation tools.
Perhaps it is hard to read. I hope I can convey it properly.
I also tried attaching the patch, but I should add that I'm inexperienced with 
python.


 > Yes. Can you confirm this will solve the code problems you reported:
 > 
https://kallithea-scm.org/repos/kallithea-incoming/changeset/088551a24485247af328f0b8e395fdbbcd62a700
 ?

1. Modify create_repo()
   I don't think this change is correct.
   I overwrote api.py and called create_repo, but 
enable_downloads/enable_statistics seems to have no effect.

   I think the call flow during API execution is as follows.

   api.py : ApiController.create_repo()
     repo.py : RepoModel.create()
   repo.py : (@celerylib.task) create_repo()

   I think that the third function does not refer to form_data, but refers to 
defs to obtain the default value and use it.
   
https://kallithea-scm.org/repos/kallithea/files/2dd317e9cc4b5ff94c1ddae7ee772d20b61259a9/kallithea/model/repo.py#L717
   I think this is probably implemented in conjunction with the web front end.

   I see two options for changes to the create_repo API.
   I think this requires a policy decision.

   Scenario 1:
     The API will no longer accept that parameter.
     Attachment: 1-scenario1.patch

   Scenario 2:
     Support parameter specification in RepoModel.
     Attachment: 1-scenario2.patch

2. Modify update_repo_group()
   I think there are two issues with parent handling.

   The first is.
   Other api parameters can basically target entities by name.
   It seems to me that as the parent_group_id parameter to RepoGroupModel.update(), the API parameter should be passed with name 
resolution instead of passing it as is.

     Attachment: 2-parent_id.patch

   The second is still unresolved and not yet known in detail.
   Only information known at this time is provided.
   The problem is that changing the parent group via the API will leave the 
database and file system in an inconsistent state.
   At that time, new_path does not seem to indicate the correct path in 
RepoGroupModel.update(). So the folder is not moved.
   It seems correct when run from the web front end. I still don't know what 
the difference is.


Supplementation.
It may not be necessary, but I will publish the docker files and scripts that I 
used in the confirmation work.
https://github.com/toras9000/kallithea_api_test



Thanks
# HG changeset patch
# User toras9000
# Date 1667574300 -32400
#  Sat Nov 05 00:05:00 2022 +0900
# Branch fix-update_repo_group
# Node ID 05ed2d77c26da61ea41f08b7ab9d83078306805a
# Parent  088551a24485247af328f0b8e395fdbbcd62a700
fix: update_repo_group api

diff --git a/kallithea/controllers/api/api.py b/kallithea/controllers/api/api.py
--- a/kallithea/controllers/api/api.py
+++ b/kallithea/controllers/api/api.py
@@ -1791,11 +1791,20 @@
   parent=None):
 repo_group = get_repo_group_or_error(repogroupid)
 
+if not group_name:
+group_name = repo_group.name
+
+parent_id = None
+if parent is not None:
+parent_group = get_repo_group_or_error(parent)
+parent_id = parent_group.group_id
+
 updates = {}
 try:
 store_update(updates, group_name, 'group_name')
 store_update(updates, description, 'group_description')
-store_update(updates, parent, 'parent_group_id')
+if parent_id is not None:
+store_update(updates, parent_id, 'parent_group_id')
 repo_group = RepoGroupModel().update(repo_group, updates)
 meta.Session().commit()
 return dict(
# HG changeset patch
# User toras9000
# Date 1667574446 -32400
#  Sat Nov 05 00:07:26 2022 +0900
# Branch fix-update_repo_group
# Node ID 1f502ad271fd32733c9242d72faba5124fcaa310
# Parent  05ed2d77c26da61ea41f08b7ab9d83078306805a
fix: update_repo_group model

diff --git a/kallithea/model/repo_group.py b/kallithea/model/repo_group.py
--- a/kallithea/model/repo_group.py
+++ b/kallithea/model/repo_group.p

Re: About API code comments and some behavior.

[toras]

Hi

Thank you for confirming the details of the content.


>> 1. Modify create_repo()
>> ...
>
> Right. Thanks. I will go with scenario2 ... but take it further and make 
repo_enable_downloads/repo_enable_statistics mandatory
> in the internal API and expose them in the UI form as well. As a part of 
that, I will also add better test coverage and clean up
> the UI forms.

That is the most preferable way to handle it. Thank you.


>> Attachment: fix-update_repo_group-2.patch
>
> Lots of things happening in that patch. Can you explain in more details what 
is changed and why ... and perhaps do it in several
> simpler changesets?

Sorry, I didn't explain myself well enough.
The change had two purposes.


The first is that RepoGroupModel.update works without specifying a group_name.
This has already been addressed in Rev.8732.
As for repo_group.parent_group and repo_group.parent_group_id, I was guessing what you pointed out, but wasn't sure, so I left 
what I had done in the original code.



The second is to modify the log output.
Sorry for the confusion, the change to the f string was just for my own viewing 
pleasure, not related to the issue.

The entities enumerated by recursive_groups_and_repos() to update the full path of the child elements are processed with log 
output, but the starting repository group has already had its entities updated before the loop.

So, for example, updating the name from 'GroupA' to 'GroupB' will result in the 
following log output
(Only the origin entity. Child elements are correct logs.)

  [kallithea.model.repo_group] Fixing group GroupB to new name GroupB

The following logs should be correct.

  [kallithea.model.repo_group] Fixing group GroupA to new name GroupB

For the purpose of logging the name before the change, the patch outputs the log before updating the entity and skips the 
originating entity in a loop process.



Thanks
--
toras9000

On 2022/12/13 5:25, Mads Kiilerich wrote:

On 30/10/2022 13:00, toras wrote:

> Yes. Can you confirm this will solve the code problems you reported:
> 
https://kallithea-scm.org/repos/kallithea-incoming/changeset/088551a24485247af328f0b8e395fdbbcd62a700
 ?

1. Modify create_repo()
  I don't think this change is correct.
  I overwrote api.py and called create_repo, but 
enable_downloads/enable_statistics seems to have no effect.

  I think the call flow during API execution is as follows.

  api.py : ApiController.create_repo()
    repo.py : RepoModel.create()
  repo.py : (@celerylib.task) create_repo()

  I think that the third function does not refer to form_data, but refers to 
defs to obtain the default value and use it.
https://kallithea-scm.org/repos/kallithea/files/2dd317e9cc4b5ff94c1ddae7ee772d20b61259a9/kallithea/model/repo.py#L717
  I think this is probably implemented in conjunction with the web front end.

  I see two options for changes to the create_repo API.
  I think this requires a policy decision.

  Scenario 1:
    The API will no longer accept that parameter.
    Attachment: 1-scenario1.patch

  Scenario 2:
    Support parameter specification in RepoModel.
    Attachment: 1-scenario2.patch



Right. Thanks. I will go with scenario2 ... but take it further and make repo_enable_downloads/repo_enable_statistics mandatory 
in the internal API and expose them in the UI form as well. As a part of that, I will also add better test coverage and clean up 
the UI forms.




2. Modify update_repo_group()
  I think there are two issues with parent handling.

  The first is.
  Other api parameters can basically target entities by name.
  It seems to me that as the parent_group_id parameter to RepoGroupModel.update(), the API parameter should be passed with 
name resolution instead of passing it as is.

    Attachment: 2-parent_id.patch



Agreed. I will do that ... but with slight changes.



  The second is still unresolved and not yet known in detail.
  The problem is that changing the parent group via the API will leave the 
database and file system in an inconsistent state.
  At that time, new_path does not seem to indicate the correct path in 
RepoGroupModel.update(). So the folder is not moved.
  It seems correct when run from the web front end. I still don't know what the 
difference is.


...


I have found the cause of the problem with the update_repo_group API behavior 
that I wrote about in my previous email.
The implementation of RepoGroupModel.update() does not seem to work correctly unless it includes a valid group_name in args, 
even if the name is not changed.

If no new name is specified for the API parameter, it may be necessary to pass 
the original name.
I have created a patch to api.py based on 088551a24485 from kallithea-imcoming.
    Attachment: fix-update_repo_group-1.patch


Agreed, but I think the problem is in RepoGroupModel.update , where it failed to update group_name to the new full path path 
after mo

Re: About API code comments and some behavior.

[toras]

Hi

Thank you for your time and taking care of this.


On 2022/12/28 5:22, Mads Kiilerich wrote:

On 18/12/2022 10:48, toras wrote:

>> Attachment: fix-update_repo_group-2.patch
>
> Lots of things happening in that patch. Can you explain in more details what is changed and why ... and perhaps do it in 
several
> simpler changesets? 

...

The second is to modify the log output.
Sorry for the confusion, the change to the f string was just for my own viewing 
pleasure, not related to the issue.



Ok. But I will just start out by making this piece of code use lazy %s 
expansion for logging like we do in all other places.


The entities enumerated by recursive_groups_and_repos() to update the full path of the child elements are processed with log 
output, but the starting repository group has already had its entities updated before the loop.

So, for example, updating the name from 'GroupA' to 'GroupB' will result in the 
following log output
(Only the origin entity. Child elements are correct logs.)

  [kallithea.model.repo_group] Fixing group GroupB to new name GroupB

The following logs should be correct.

  [kallithea.model.repo_group] Fixing group GroupA to new name GroupB

For the purpose of logging the name before the change, the patch outputs the log before updating the entity and skips the 
originating entity in a loop process.



Right - thanks for clarifying. I applied fix doing just that, but also made sure we don't do excessive logging when parameters 
specified without changing.



Thanks for the contribution.

/Mads


--
toras9000

___
kallithea-general mailing list
kallithea-general@sfconservancy.org
https://lists.sfconservancy.org/mailman/listinfo/kallithea-general

About permission evaluation for repository group owner.

[toras]

Hi.


Commit abc29122c7f2 has been addressed to allow repository group owner changes.
I think the owner change itself is working.
However, for non-admin users, the permission evaluation in the repository group 
seems to be incorrect.

For example, if you try to create a repository in that repository group as a changed owner user, you will get the error 'no 
permission to create repo in '.
After a little research, it seemed to me that the repository_group_permissions() in auth.py, which is used beyond the 
HasRepoGroupPermissionLevel() call, needs to be evaluated for being the owner of the repository group.

Could you please confirm this?


Additionally, I have a question regarding the permission evaluation for 
repository groups, separate from the issue mentioned above.
Currently, regular users cannot create repositories within a repository group unless they have administrative privileges for the 
group.

I feel that requiring administrative privileges is a bit excessive.
What are your thoughts on this matter?



Thanks

--
toras9000


___
kallithea-general mailing list
kallithea-general@sfconservancy.org
https://lists.sfconservancy.org/mailman/listinfo/kallithea-general

Re: About permission evaluation for repository group owner.

[toras]

Hi.


Thank you for confirming.

> I propose 
https://kallithea-scm.org/repos/kallithea-incoming/changeset/dee1b60bad29621882eb769eb5bc8707647ccf1d
 .

As far as I have tried, I believe this change fixes the new owner to operate 
correctly. (Both from the web and from the API.)


> I propose 
https://kallithea-scm.org/repos/kallithea-incoming/changeset/bf7369172810fb1a9452af767a2168edba3dc2f3

I believe that this change is also necessary to properly remove permissions 
from the previous owner.


> Do you see other problems related to these changes? Any other places where 
the code makes incorrect assumptions on repo groups
> and owner / permissions?

Related to the second issue, there seems to be a problem that "the owner (non-super user) of a group cannot set permissions for 
himself/herself".

In the permission settings screen, the owner cannot set the following write 
permissions for himself/herself.
Any attempt to do so fails with the message 'Cannot revoke permission for 
yourself as admin'.
I think this is part of the behavior that remains from when we were handling explicitly granting administrative privileges to 
groups.


However, some groups can be modified, and there may be conditions under which 
the above failure occurs.
This may be the case for groups created by ordinary users themselves.


> If you edit a repository group, the permissions tab will describe it as "Write" as 
"(Add repos)". Admin access should not be
> necessary. Please verify that you really see the behaviour you describe.
> (For some reason, repo group creation is more constrained in than repo 
creation... but that's yet another story.)

My apologies.
I meant to write "repository groups" instead of "repository" but I was wrong.

Sometimes I wonder why, because I want to create a group with the following 
structure, but cannot do so with only write permission.

personals <- Create by admin.
  + userA_group   <- Create by userA.
  + userB_group   <- Create by userB.


Thanks

--
toras9000
___
kallithea-general mailing list
kallithea-general@sfconservancy.org
https://lists.sfconservancy.org/mailman/listinfo/kallithea-general

Re: About permission evaluation for repository group owner.

[toras]

Hi

> Right - nice catch. I don't think there are any valid use cases for this code 
now. And there is also similar code in the web
> templates.
>
> Please consider 
https://kallithea-scm.org/repos/kallithea-incoming/changeset/ab8e9f05241a .

Thanks for the correction code.
I have applied this change and verified that the permissions can be set 
successfully.


>> (repository group permission)
>> ...
> It could perhaps be changed, but that would be a different discussion, and 
not suitable for the stable branch.

Agreed.
There may be room for improvement, but I don't think it is something to be done 
hastily.


Thanks

--
toras9000

___
kallithea-general mailing list
kallithea-general@sfconservancy.org
https://lists.sfconservancy.org/mailman/listinfo/kallithea-general

Proposal for extra field support in API

[toras]

Hi.

This is a proposal to add extra field support in the API.

I am not sure if this fits the project's policy, as the content was changed for 
personal purposes.
If it doesn't fit, you can reject it.

There are two changes.

01-api update extra field by update_repo.patch
  This is a change to accept updates to extra fields at the updat_repo endpoint.
  I recently found that the model already supports updates.
  However, the API interface was not accepted, so this is to add parameters for 
it.

  A point for a little consideration is the method of specifying key names for 
extra fields.
  For example, the get_repo API uses a flat hierarchy of properties with an 
'ex_' prefix to retrieve information.
  This proposed change is not symmetrical with it. However, I believe this 
format is more convenient.

02-api add endpoint for extra fields.patch
  It adds endpoints for retrieving, creating, and deleting extra field 
definitions in the repository.
  I am not sure if the way db is referenced here, etc., is appropriate for the 
policy.


In both of the above, I included methods for testing.
However, we do not know if this way of creating the test is also appropriate.


Thanks

--
toras9000

# HG changeset patch
# User toras9000 
# Date 1691408559 -32400
#  Mon Aug 07 20:42:39 2023 +0900
# Branch stable
# Node ID 6f59d9f02044092bac1993879dcea4eece45d049
# Parent  36a36ebdf4bbc4da77c41cabdbdf4a688e8fbeea
api: update extra field by update_repo

diff --git a/kallithea/controllers/api/api.py b/kallithea/controllers/api/api.py
--- a/kallithea/controllers/api/api.py
+++ b/kallithea/controllers/api/api.py
@@ -1017,7 +1017,8 @@
 description=None, private=None,
 clone_uri=None, landing_rev=None,
 enable_statistics=None,
-enable_downloads=None):
+enable_downloads=None,
+extra_fields=None):
 """
 Updates repo
 """
@@ -1037,6 +1038,11 @@
 'Only Kallithea admin can specify `owner` param'
 )
 
+if extra_fields is not None:
+ex_field_setting = db.Setting.get_by_name('repository_fields')
+if (ex_field_setting is None) or (not 
ex_field_setting.app_settings_value):
+raise JSONRPCError('Extra field setting is disabled.')
+
 updates = {}
 repo_group = group
 if repo_group is not None:
@@ -1056,6 +1062,11 @@
 store_update(updates, enable_statistics, 'repo_enable_statistics')
 store_update(updates, enable_downloads, 'repo_enable_downloads')
 
+if isinstance(extra_fields, dict):
+for key, val in extra_fields.items():
+if (key is not None) and (val is not None):
+store_update(updates, val, db.RepositoryField.PREFIX + 
key)
+
 RepoModel().update(repo, **updates)
 meta.Session().commit()
 return dict(
diff --git a/kallithea/tests/api/api_base.py b/kallithea/tests/api/api_base.py
--- a/kallithea/tests/api/api_base.py
+++ b/kallithea/tests/api/api_base.py
@@ -1243,6 +1243,72 @@
 finally:
 fixture.destroy_repo(repo_name)
 
+def test_api_update_repo_extra_field_change(self):
+repo_name = 'admin_owned'
+repo = fixture.create_repo(repo_name, repo_type=self.REPO_TYPE)
+try:
+RepoModel().grant_user_permission(repo=repo_name, 
user=self.TEST_USER_LOGIN, perm='repository.admin')
+db.Setting.create_or_update('repository_fields', True, 'bool') # 
extra_fields enabled
+fixture.create_repo_extra_field(repo, field_key='testkey1', 
field_value='testval1')
+fixture.create_repo_extra_field(repo, field_key='testkey2', 
field_value='testval2')
+
+expected = {
+'msg': 'updated repo ID:%s %s' % (repo.repo_id, repo_name),
+'repository': repo.get_api_data()
+}
+expected['repository']['ex_testkey1'] = 'testval1'
+expected['repository']['ex_testkey2'] = 'changeval'
+
+updates = { 'extra_fields': { 'testkey2': 'changeval', }, }
+id_, params = _build_data(self.apikey_regular, 'update_repo', 
repoid=repo_name, **updates)
+response = api_call(self, params)
+
+self._compare_ok(id_, expected, given=response.body)
+finally:
+fixture.destroy_repo(repo_name)
+
+def test_api_update_repo_extra_field_missing(self):
+repo_name = 'admin_owned'
+repo = fixture.create_repo(repo_name, repo_type=self.REPO_TYPE)
+try:
+RepoModel().grant_user_permission(repo=repo_name, 
user=self.TEST_USER_LOGIN, perm='repository.admin')
+db.Setting.create_or_update('repository_fields', True, 'bool') # 
extra_fields enabled
+fixture.create_repo_extra_field(repo, field_key='testkey1', 
field