[Pywikipedia-bugs] [Maniphest] [Commented On] T364393: pywikibot does not handle properly 502 Server Error reading pages
Masti added a comment. In T364393#9783360 <https://phabricator.wikimedia.org/T364393#9783360>, @Xqt wrote: > @Masti: I assumed the -start:! tested it on several wikis but didn’t run into this issue. I made a patch and both server errors should lead to retry loops. > > I still think this has something to do with the database conversion of MW 1.43: some api requests are very lame or fails with timeout or server error. I think so. But can we handle the timeouts in pw? As the other erros from API are handled with incremental delays and retry. TASK DETAIL https://phabricator.wikimedia.org/T364393 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Xqt, Masti Cc: JJMC89, matej_suchanek, Xqt, Aklapper, Masti, pywikibot-bugs-list, mevo, Isabelladantes1983, Themindcoder, Adamm71, Jersione, Hellket777, LisafBia6531, 786, Biggs657, PotsdamLamb, Jyoo1011, JohnsonLee01, Juan90264, SHEKH, Dijkstra, Alter-paule, Beast1978, Un1tY, Khutuck, Zkhalido, Hook696, Kent7301, joker88john, Viztor, CucyNoiD, Wenyi, Gaboe420, Giuliamocci, Cpaulf30, Af420, Bsandipan, Tbscho, MayS, Lewizho99, Mdupont, Maathavan, Dvorapa, Altostratus, Neuronton, Avicennasis, mys_721tx, jayvdb, Alchimista ___ pywikibot-bugs mailing list -- pywikibot-bugs@lists.wikimedia.org To unsubscribe send an email to pywikibot-bugs-le...@lists.wikimedia.org
[Pywikipedia-bugs] [Maniphest] [Commented On] T364393: pywikibot does not handle properly 502 Server Error reading pages
Masti added a comment.
In T364393#9782642 <https://phabricator.wikimedia.org/T364393#9782642>, @Xqt
wrote:
> @Masti: What was your pagegenerator option when running your script?
-start:'!'
this and similar scripts sometimes run OK. but crash from time to time.
BTW: sometimes the server error is 503 like:
ERROR: Traceback (most recent call last):
File "/home/masti/pw/core/pywikibot/data/api/_requests.py", line 684, in
_http_request
response = http.request(self.site, uri=uri,
File "/home/masti/pw/core/pywikibot/comms/http.py", line 283, in request
r = fetch(baseuri, headers=headers, **kwargs)
File "/home/masti/pw/core/pywikibot/comms/http.py", line 457, in fetch
callback(response)
File "/home/masti/pw/core/pywikibot/comms/http.py", line 354, in
error_handling_callback
raise ServerError(
pywikibot.exceptions.ServerError: 503 Server Error: Service Unavailable
Traceback (most recent call last):
File "/home/masti/pw/core/pwb.py", line 40, in
sys.exit(main())
File "/home/masti/pw/core/pwb.py", line 36, in main
runpy.run_path(str(path), run_name='__main__')
File "/usr/lib/python3.10/runpy.py", line 289, in run_path
return _run_module_code(code, init_globals, run_name,
File "/usr/lib/python3.10/runpy.py", line 96, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/masti/pw/core/pywikibot/scripts/wrapper.py", line 521, in
main()
File "/home/masti/pw/core/pywikibot/scripts/wrapper.py", line 505, in main
if not execute():
File "/home/masti/pw/core/pywikibot/scripts/wrapper.py", line 492, in
execute
run_python_file(filename, script_args, module)
File "/home/masti/pw/core/pywikibot/scripts/wrapper.py", line 149, in
run_python_file
exec(compile(source, filename, 'exec', dont_inherit=True),
File "masti/ms-contains.py", line 433, in
main()
File "masti/ms-contains.py", line 427, in main
bot.run() # guess what it does
File "masti/ms-contains.py", line 147, in run
for page in self.generator:
File "/home/masti/pw/core/pywikibot/pagegenerators/__init__.py", line 660,
in PreloadingGenerator
yield from site.preloadpages(group, groupsize=groupsize,
File "/home/masti/pw/core/pywikibot/site/_generators.py", line 202, in
preloadpages
for pagedata in rvgen:
File "/usr/lib/python3.10/_collections_abc.py", line 330, in __next__
return self.send(None)
File "/home/masti/pw/core/pywikibot/tools/collections.py", line 279, in send
return next(self._started_gen)
File "/home/masti/pw/core/pywikibot/data/api/_generators.py", line 781, in
generator
yield from super().generator
File "/home/masti/pw/core/pywikibot/data/api/_generators.py", line 607, in
generator
self.data = self.request.submit()
File "/home/masti/pw/core/pywikibot/data/api/_requests.py", line 993, in
submit
response, use_get = self._http_request(use_get, uri, body, headers,
File "/home/masti/pw/core/pywikibot/data/api/_requests.py", line 684, in
_http_request
response = http.request(self.site, uri=uri,
File "/home/masti/pw/core/pywikibot/comms/http.py", line 283, in request
r = fetch(baseuri, headers=headers, **kwargs)
File "/home/masti/pw/core/pywikibot/comms/http.py", line 457, in fetch
callback(response)
File "/home/masti/pw/core/pywikibot/comms/http.py", line 354, in
error_handling_callback
raise ServerError(
pywikibot.exceptions.ServerError: 503 Server Error: Service Unavailable
CRITICAL: Exiting due to uncaught exception ServerError: 503 Server Error:
Service Unavailable
TASK DETAIL
https://phabricator.wikimedia.org/T364393
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Xqt, Masti
Cc: JJMC89, matej_suchanek, Xqt, Aklapper, Masti, pywikibot-bugs-list, mevo,
Isabelladantes1983, Themindcoder, Adamm71, Jersione, Hellket777, LisafBia6531,
786, Biggs657, PotsdamLamb, Jyoo1011, JohnsonLee01, Juan90264, SHEKH, Dijkstra,
Alter-paule, Beast1978, Un1tY, Khutuck, Zkhalido, Hook696, Kent7301,
joker88john, Viztor, CucyNoiD, Wenyi, Gaboe420, Giuliamocci, Cpaulf30, Af420,
Bsandipan, Tbscho, MayS, Lewizho99, Mdupont, Maathavan, Dvorapa, Altostratus,
Neuronton, Avicennasis, mys_721tx, jayvdb, Alchimista
___
pywikibot-bugs mailing list -- pywikibot-bugs@lists.wikimedia.org
To unsubscribe send an email to pywikibot-bugs-le...@lists.wikimedia.org
[Pywikipedia-bugs] [Maniphest] [Commented On] T364393: pywikibot does not handle properly 502 Server Error reading pages
Masti added a comment. I increased the timeout to 120s but it crashed again with same error. TASK DETAIL https://phabricator.wikimedia.org/T364393 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Masti Cc: Xqt, Aklapper, Masti, pywikibot-bugs-list, mevo, PotsdamLamb, Jyoo1011, JohnsonLee01, SHEKH, Dijkstra, Khutuck, Zkhalido, Viztor, Wenyi, Tbscho, MayS, Mdupont, JJMC89, Dvorapa, Altostratus, Avicennasis, mys_721tx, jayvdb, Alchimista ___ pywikibot-bugs mailing list -- pywikibot-bugs@lists.wikimedia.org To unsubscribe send an email to pywikibot-bugs-le...@lists.wikimedia.org
[Pywikipedia-bugs] [Maniphest] [Commented On] T364393: pywikibot does not handle properly 502 Server Error reading pages
Masti added a comment. I will test and come back with the answer. Unfortunately it is not easy replicated so one have to wait till it happens. TASK DETAIL https://phabricator.wikimedia.org/T364393 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Masti Cc: Xqt, Aklapper, Masti, pywikibot-bugs-list, mevo, PotsdamLamb, Jyoo1011, JohnsonLee01, SHEKH, Dijkstra, Khutuck, Zkhalido, Viztor, Wenyi, Tbscho, MayS, Mdupont, JJMC89, Dvorapa, Altostratus, Avicennasis, mys_721tx, jayvdb, Alchimista ___ pywikibot-bugs mailing list -- pywikibot-bugs@lists.wikimedia.org To unsubscribe send an email to pywikibot-bugs-le...@lists.wikimedia.org
[Pywikipedia-bugs] [Maniphest] [Created] T364393: pywikibot does not handle properly 502 Server Error reading pages
Masti created this task.
Masti added a project: Pywikibot.
Restricted Application added subscribers: pywikibot-bugs-list, Aklapper.
TASK DESCRIPTION
while running pywikibot on pl.wikipedia I quite often encounter code crashes
after page generator encounters 502 Server Error from wikimedia API
**What happens?**:
script throws exception and crashes
Retrieving 50 pages from wikipedia:pl.
ERROR: Traceback (most recent call last):
File "/home/masti/pw/core/pywikibot/data/api/_requests.py", line 684, in
_http_request
response = http.request(self.site, uri=uri,
File "/home/masti/pw/core/pywikibot/comms/http.py", line 283, in request
r = fetch(baseuri, headers=headers, **kwargs)
File "/home/masti/pw/core/pywikibot/comms/http.py", line 457, in fetch
callback(response)
File "/home/masti/pw/core/pywikibot/comms/http.py", line 354, in
error_handling_callback
raise ServerError(
pywikibot.exceptions.ServerError: 502 Server Error: Server Hangup
Traceback (most recent call last):
File "/home/masti/pw/core/pwb.py", line 40, in
sys.exit(main())
File "/home/masti/pw/core/pwb.py", line 36, in main
runpy.run_path(str(path), run_name='__main__')
File "/usr/lib/python3.10/runpy.py", line 289, in run_path
return _run_module_code(code, init_globals, run_name,
File "/usr/lib/python3.10/runpy.py", line 96, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/masti/pw/core/pywikibot/scripts/wrapper.py", line 521, in
main()
File "/home/masti/pw/core/pywikibot/scripts/wrapper.py", line 505, in main
if not execute():
File "/home/masti/pw/core/pywikibot/scripts/wrapper.py", line 492, in
execute
run_python_file(filename, script_args, module)
File "/home/masti/pw/core/pywikibot/scripts/wrapper.py", line 149, in
run_python_file
exec(compile(source, filename, 'exec', dont_inherit=True),
File "masti/ms-contains.py", line 433, in
main()
File "masti/ms-contains.py", line 427, in main
bot.run() # guess what it does
File "masti/ms-contains.py", line 147, in run
for page in self.generator:
File "/home/masti/pw/core/pywikibot/pagegenerators/__init__.py", line 660,
in PreloadingGenerator
yield from site.preloadpages(group, groupsize=groupsize,
File "/home/masti/pw/core/pywikibot/site/_generators.py", line 202, in
preloadpages
for pagedata in rvgen:
File "/usr/lib/python3.10/_collections_abc.py", line 330, in __next__
return self.send(None)
File "/home/masti/pw/core/pywikibot/tools/collections.py", line 279, in send
return next(self._started_gen)
File "/home/masti/pw/core/pywikibot/data/api/_generators.py", line 781, in
generator
yield from super().generator
File "/home/masti/pw/core/pywikibot/data/api/_generators.py", line 607, in
generator
self.data = self.request.submit()
File "/home/masti/pw/core/pywikibot/data/api/_requests.py", line 993, in
submit
response, use_get = self._http_request(use_get, uri, body, headers,
File "/home/masti/pw/core/pywikibot/data/api/_requests.py", line 684, in
_http_request
response = http.request(self.site, uri=uri,
File "/home/masti/pw/core/pywikibot/comms/http.py", line 283, in request
r = fetch(baseuri, headers=headers, **kwargs)
File "/home/masti/pw/core/pywikibot/comms/http.py", line 457, in fetch
callback(response)
File "/home/masti/pw/core/pywikibot/comms/http.py", line 354, in
error_handling_callback
raise ServerError(
pywikibot.exceptions.ServerError: 502 Server Error: Server Hangup
CRITICAL: Exiting due to uncaught exception ServerError: 502 Server Error:
Server Hangup
**What should have happened instead?**:
wait for server to come back as the errors are temporary
**Software version** (on `Special:Version` page; skip for WMF-hosted wikis
like Wikipedia):
Pywikibot: [https] masti01-pywikibot.git (04383ba
<https://phabricator.wikimedia.org/rPWBC04383baed7b64bb9ca0152e8147df8ff80d269b0>,
g18666, 2024/05/06, 16:04:47, master)
Release version: 9.2.0.dev2
packaging version: 24.0
mwparserfromhell version: 0.6.6
wikitextparser version: n/a
requests version: 2.31.0
cacerts:
/home/masti/pw/core/venv/lib/python3.10/site-packages/certifi/cacert.pem
certificate test: ok
Python: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
**Other information** (browser name/version, screenshots, etc.):
TASK DETAIL
https://phabricator.wikimedia.org/T364393
EMAIL PREFERENCES
http
[pywikibot] Re: Bots and tools need to upgrade to Pywikibot 6.6.1
one thing that falls to my mind: there are a lot of changes and breaking changes. But please keep in mind tha ta lot of bots are run by people that do it in their free time. They are not focused on doing that as their primary job. Please try to keep pywikibot updates in a way that allows for them to keep up. I have bots that run on very outdated code and they do run fine. That means that forcing updates is not necessary. masti On 9/22/21 8:53 PM, Kunal Mehta wrote: Hi everyone, Bots and tools using Pywikibot must upgrade to version 6.6.1[1] otherwise they will break when deprecated API parameters are removed[2]. If you have any questions or need help in upgrading, please reach out using one of the Pywikibot communication channels[3]. [1] https://doc.wikimedia.org/pywikibot/stable/changelog.html [2] https://www.mediawiki.org/wiki/MediaWiki_1.37/Deprecation_of_legacy_API_token_parameters [3] https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Pywikibot/Communication Thanks, -- Legoktm ___ pywikibot mailing list -- pywikibot@lists.wikimedia.org To unsubscribe send an email to pywikibot-le...@lists.wikimedia.org ___ pywikibot mailing list -- pywikibot@lists.wikimedia.org To unsubscribe send an email to pywikibot-le...@lists.wikimedia.org
[Pywikipedia-bugs] [Maniphest] [Commented On] T277607: unable to login due to CAPTCHA while switching users
Masti added a comment. Those bots are autoconfirmed. .admin bot has even an admin flag TASK DETAIL https://phabricator.wikimedia.org/T277607 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Masti Cc: Aklapper, pywikibot-bugs-list, Masti, Jyoo1011, JohnsonLee01, SHEKH, Dijkstra, Khutuck, Zkhalido, Viztor, Wenyi, Tbscho, MayS, Mdupont, JJMC89, Dvorapa, Altostratus, Avicennasis, mys_721tx, jayvdb, Alchimista ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Retitled] T277607: unable to login due to CAPTCHA while switching users
Masti renamed this task from "unable to login du to CAPTCHA while switching users" to "unable to login due to CAPTCHA while switching users". TASK DETAIL https://phabricator.wikimedia.org/T277607 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Masti Cc: Aklapper, pywikibot-bugs-list, Masti, Jyoo1011, JohnsonLee01, SHEKH, Dijkstra, Khutuck, Zkhalido, Viztor, Wenyi, Tbscho, MayS, Mdupont, JJMC89, Dvorapa, Altostratus, Avicennasis, mys_721tx, jayvdb, Alchimista ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Created] T277607: unable to login du to CAPTCHA while swithching users
Masti created this task.
Masti added a project: Pywikibot.
Restricted Application added subscribers: pywikibot-bugs-list, Aklapper.
TASK DESCRIPTION
I run scripts with 2 accounts: bot and admin bot
Both accounts have botpasswords defined
$ python3 pwb.py login
Logged in on wikipedia:pl as MastiBot.admin.
(venv) [mst@mort core]$ python3 pwb.py login -user:mastiBot
Logging in to wikipedia:pl as MastiBot@mastiBot
Traceback (most recent call last):
File "/home/mst/pw3/core/pywikibot/data/api.py", line 2990, in login_to_site
response['messagecode'] == 'login-throttled'
KeyError: 'messagecode'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "pwb.py", line 363, in
if not main():
File "pwb.py", line 358, in main
file_package)
File "pwb.py", line 75, in run_python_file
main_mod.__dict__)
File "./scripts/login.py", line 151, in
main()
File "./scripts/login.py", line 139, in main
site.login(autocreate=autocreate)
File "/home/mst/pw3/core/pywikibot/tools/__init__.py", line 1478, in wrapper
return obj(*__args, **__kw)
File "/home/mst/pw3/core/pywikibot/site/_apisite.py", line 404, in login
if login_manager.login(retry=True, autocreate=autocreate):
File "/home/mst/pw3/core/pywikibot/login.py", line 312, in login
self.login_to_site()
File "/home/mst/pw3/core/pywikibot/data/api.py", line 2998, in login_to_site
'handled:\n{}'.format(response))
pywikibot.exceptions.CaptchaError: Captcha encountered which cannot be
handled:
{'result': 'Aborted', 'reason': 'Cannot log in when using
MediaWiki\\Session\\BotPasswordSessionProvider sessions.'}
CRITICAL: Exiting due to uncaught exception
$python --version
Python 3.6.8
$ python3 pwb.py version
Pywikibot: [g...@github.com:masti0] (93ca9c1, g14433, 2021/03/16, 15:48:38,
OUTDATED)
Release version: 6.1.0.dev0
requests version: 2.25.1
cacerts:
/home/mst/pw3/core/venv/lib64/python3.6/site-packages/certifi/cacert.pem
certificate test: ok
Python: 3.6.8 (default, Aug 13 2020, 07:46:32)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]
PYWIKIBOT_DIR: Not set
PYWIKIBOT_DIR_PWB:
PYWIKIBOT_NO_USER_CONFIG: Not set
Config base dir: /home/mst/pw3/core
Usernames for family "wikipedia":
*: MastiBot.admin
on other machine "main" user is mastiBot and situation is exactly same but
with reversed users
TASK DETAIL
https://phabricator.wikimedia.org/T277607
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Masti
Cc: Aklapper, pywikibot-bugs-list, Masti, Jyoo1011, JohnsonLee01, SHEKH,
Dijkstra, Khutuck, Zkhalido, Viztor, Wenyi, Tbscho, MayS, Mdupont, JJMC89,
Dvorapa, Altostratus, Avicennasis, mys_721tx, jayvdb, Alchimista
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
Re: [pywikibot] Getting started with pywikibot
Hi Larry, ad 2. yes. use -cat:categoryName or -catr:categoryName as generator. look in pagegenarators.py script. It gives all generators you can use when starting script. masti On 19.08.2020 01:39, Larry Martell wrote: On Mon, Aug 17, 2020 at 11:43 PM wrote: Hi Larry, there are two way to use Pywikibot as a side package: 1. add a user-config.py in your /variant folder. A minimal sample can be found at https://gerrit.wikimedia.org/r/plugins/gitiles/pywikibot/core/+/88f22977fd186d888c4d9e8cbac11d76e87bd607/user-config.py.sample 2. or set a PYWIKIBOT_NO_USER_CONFIG environment variable to 1 or 2 to ignore user-config settings The pwb.py script wrapper (as well as the generate_... scripts are available in directory mode only. You can get it including predefined scripts from nightly https://pywikibot.toolforge.org/ or as git repository from gerrit or from github clone https://github.com/wikimedia/pywikibot Adding pwb.py and generate_user_files.py/generate_family_file.py to side oackage is still under development https://gerrit.wikimedia.org/r/c/pywikibot/core/+/560057 Seem our documentation is insufficient, sorry for that. If you have further questions do not hesitate to ask. Thanks for the reply - this was very helpful. A couple more questions: 1. Is there any way to login with MFA? Our wiki requires either choosing a number from an authenticator app or using a userid and password and then entering a code which was texted. 2. Is there a way to get all pages within a category? Thanks! Larry Am 17.08.2020 um 21:41 schrieb Larry Martell : We have an internal wiki and I want to use pywikibot to get some data from it. Installed it from pip but when I import it I get: import pywikibot Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python3.7/site-packages/pywikibot/__init__.py", line 25, in from pywikibot.bot import ( File "/usr/local/lib/python3.7/site-packages/pywikibot/bot.py", line 105, in from pywikibot import config2 as config File "/usr/local/lib/python3.7/site-packages/pywikibot/config2.py", line 373, in base_dir = get_base_dir() File "/usr/local/lib/python3.7/site-packages/pywikibot/config2.py", line 367, in get_base_dir raise RuntimeError(exc_text) RuntimeError: No user-config.py found in directory '/vagrant'. Looking here https://www.mediawiki.org/wiki/Manual:Pywikibot/Installation#Configure_Pywikibot I see a reference to pwb.py, but that does not exist in /usr/local/lib/python3.7/site-packages/pywikibot. What am I missing here? Thanks! ___ pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot ___ pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot ___ pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot ___ pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
Re: [pywikibot] checking links against SpamFilter
thanks,
that's how I've done it. I tought there is maybe an ready class.
masti
On 24.10.2020 13:44, John wrote:
https://en.wikipedia.org/w/api.php?action="">
On Sat, Oct 24, 2020 at 2:29
AM masti <mast...@gmail.com> wrote:
Hi,
is there a way to check if the link will be catched by
SpamFilter
without saving the page?
I am doing some link statistics and cannot save the results
page due to
global SpamFilter.
masti
___
pywikibot mailing list
pywikibot@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot
___
pywikibot mailing list
pywikibot@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot
___
pywikibot mailing list
pywikibot@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot
[pywikibot] checking links against SpamFilter
Hi, is there a way to check if the link will be catched by SpamFilter without saving the page? I am doing some link statistics and cannot save the results page due to global SpamFilter. masti ___ pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
[Wikimedia-l] TOR- vs. ŁukaszGarczewski (WMPL)
Łukasz, myślę, ze warto by było, żebyś edytował, w zakresie dyrektorskim, z konta służbowego, tak jak pracownicy czyli jednak Łukasz Garczewski (WMPL) a nie TOR. Starzy wikipedysci wiedzą, ale dla czystości edycji myślę, że warto zmienić. masti ___ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l New messages to: Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe>
Re: [Wikimedia-l] Universal Code of Conduct Drafting Committee - Call for participation
Could You evaluate how You came to the firm NO? I cannot see it from the link You provided masti On 31.07.2020 17:37, Todd Allen wrote: It was a "no" to having any type of universal code of conduct. You can see the discussion here: https://meta.wikimedia.org/wiki/Talk:Strategy/Wikimedia_movement/2018-20/Recommendations/Iteration_1/Diversity/9#Terms_of_Use_for_the_WMF On Fri, Jul 31, 2020 at 8:57 AM Dan Garry (Deskana) wrote: On Fri, 31 Jul 2020 at 15:29, Todd Allen wrote: That was a firm "No" on any Universal Code of Conduct. There shouldn't be a "drafting committee" for it, it was disapproved. It's not clear to me what you're referring to here. What is the "that" that was a "firm no"? Dan ___ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l New messages to: Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe> ___ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l New messages to: Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe> ___ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l New messages to: Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe>
Re: [Wikimedia-l] Commons
commons problem is a hostile admin environment. people without understnd for non UK/US ways of handling copyright law technical issues are important, as new users do not know how to do it. but once they overcome that heir pictures got deleted masti On 17.05.2020 05:04, Benjamin Ikuta wrote: Anecdotally, it seems people sometimes don't upload their photos to Commons because they don't realize that the scope of Commons is much broader than that of Wikipedia. Has there been, or should there be, any research into this, or why people don't contribute more broadly? ~Benjamin ___ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l New messages to: Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe> ___ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l New messages to: Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe>
[pywikibot] WikiData read constraints
Is there any outlook about when WikiData API will be working again? I can understand throttling write operations but my bots are struggling to get simple info lie a template interwikis :( Retrieving 1 pages from wikipedia:pl. Treating: Szablon:CEE Spring 2020 Sleeping for 5.0 seconds, 2020-05-22 15:05:42 Sleeping for 5.0 seconds, 2020-05-22 15:05:47 Sleeping for 5.0 seconds, 2020-05-22 15:05:52 Sleeping for 6.5 seconds, 2020-05-22 15:05:58 Sleeping for 7.1 seconds, 2020-05-22 15:06:04 Sleeping for 8.6 seconds, 2020-05-22 15:06:12 Sleeping for 10.0 seconds, 2020-05-22 15:06:20 Sleeping for 11.4 seconds, 2020-05-22 15:06:31 Sleeping for 12.9 seconds, 2020-05-22 15:06:42 Sleeping for 14.3 seconds, 2020-05-22 15:06:56 masti ___ pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
[Pywikipedia-bugs] [Maniphest] [Commented On] T242081: Pywikibot fails to access Wikidata due to high maxlag lately
Masti added a comment. the problem is not ob pywikibot side but rather on Wikidata side. Wikidata API is not able to handle the traffic. TASK DETAIL https://phabricator.wikimedia.org/T242081 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Xqt, Masti Cc: Masti, THE_IT, SilentSpike, Aschroet, Multichill, Xqt, Ladsgroup, Strainu, Aklapper, pywikibot-bugs-list, Dvorapa, Blissjay007, Oblanco79, Alter-paule, Beast1978, Un1tY, Zkhalido, Hook696, Daryl-TTMG, RomaAmorRoma, E.S.A-Sheild, Kent7301, Meekrab2012, joker88john, Viztor, CucyNoiD, NebulousIris, Wenyi, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Af420, Darkminds3113, Bsandipan, Lordiis, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, Tbscho, MayS, WSH1906, Lewizho99, Mdupont, JJMC89, Maathavan, Altostratus, Avicennasis, mys_721tx, jayvdb, Alchimista, Rxy, Krenair ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
Re: [List admins] Invitation to test mailing list mirroring and emulation in Wikimedia Space
+1 webbased solutions are not good equivalent to mailing list. The latter are much more flexible. So I also would rather see mailing list migration to someting newer than Mailman 2 than move to a webbased solution masti On 29.10.2019 19:27, MA wrote: Hi, I feel the same as Risker and think we should instead investigate how to upgrade our mailing lists to Mailman 3. Regards, M. El mar., 29 oct. 2019 18:27, Risker <risker...@gmail.com> escribió: Hi Quim - Perhaps it would be helpful if you were to identify what types of lists are suitable for mirroring, and what happens when they are mirrored, because that is not clear from your post. (And no, I don't want to have to have *another* software experience just to ask these questions.) For example: Are all mirrored lists publicly accessible and viewable? Is there a way to control even read-only access? Are private mailing lists suitable for Discourse? Is everything archived? If so, how, and who controls them? Can archives be modified, and if so, by whom? How does editing/responding to mailing list posts on the Discourse mirror get reflected in the main mailing list? Can lists have individual guidelines that are different from the "common" ones? I will note that all of the lists I administer are *not* public lists, and several are non-archiving. They each have different guidelines as to what should and should not be included and/or discussed on them. Risker/Anne On Tue, 29 Oct 2019 at 13:02, Quim Gil <q...@wikimedia.org> wrote: Hi, I have just published a proposal to start testing mailing list mirroring and emulation at the Discourse instance we have at Wikimedia Space. If you are interested in discussing and exploring potential post-Mailman scenarios, join in. Check the proposal at https://discuss-space.wmflabs.org/t/testing-the-path-from-mailing-lists-to-wikimedia-space/1934. Feedback is welcome there or here. :) If you want to propose a mailing list to mirror or you are personally interested in testing mailing list emulation, let's talk. -- Quim Gil (he/him) Senior Manager of Community Relations @ Wikimedia Foundation https://meta.wikimedia.org/wiki/User:Qgil-WMF ___ Listadmins mailing list Listadmins@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/listadmins By sending a message to this list, you email all admins of all lists. To request technical changes for a specific list, instead create a task in Phabricator. See https://meta.wikimedia.org/wiki/Mailing_lists ___ Listadmins mailing list Listadmins@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/listadmins By sending a message to this list, you email all admins of all lists. To request technical changes for a specific list, instead create a task in Phabricator. See https://meta.wikimedia.org/wiki/Mailing_lists ___ Listadmins mailing list Listadmins@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/listadmins By sending a message to this list, you email all admins of all lists. To request technical chang
[pywikibot] getInterwiki
It used to be a getInterwiki method before Wikidata era. What is the easiest way to get them instead of parsing sitelinks list of WD object? masti ___ pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
Re: [Wikimedia-l] Wikimedia Social: non-profit social networking service ?
You forgot Discord https://en.wikipedia.org/wiki/Discord_(software) that used to be a voice app but has a very nice text feauters also masti On 09.08.2018 06:40, mathieu lovato stumpf guntz wrote: Hello, Following a discussion on a Wikisource Telegram group, I searched a bit about mastodon and Wikimedia, and I found back this current thread[1] as well as the following - https://phabricator.wikimedia.org/T198363 - pointing to https://mastodon.technology/@danielhglus/100278498498332671 - evoking a conversation on WP:VPIL, that is actually (most likely) refering to https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(idea_lab)/Archive_25#Mastodon_instance? - https://meta.wikimedia.org/wiki/Social_media - https://meta.wikimedia.org/wiki/Social_networking I was initially looking for something able to do at least the same as Telegram groups, with at least the same cross-devices ease of use. With what I played so far on Mastodon, I don't think it would fulfill the same feature set as Telegram, but I was already suggested a few other solution to further this inquiry: - Signal https://signal.org/ https://en.wikipedia.org/wiki/Signal_(software) <https://en.wikipedia.org/wiki/Signal_%28software%29> - Ring https://en.wikipedia.org/wiki/Ring_(software) <https://en.wikipedia.org/wiki/Ring_%28software%29> - Wire https://wire.com https://en.wikipedia.org/wiki/Wire_(software) <https://en.wikipedia.org/wiki/Wire_%28software%29> - Tox https://en.wikipedia.org/wiki/Tox_%28protocol%29 - Matrix http://matrix.org/ https://en.wikipedia.org/wiki/Matrix_(communication_protocol) <https://en.wikipedia.org/wiki/Matrix_%28communication_protocol%29> My experience with the two former don't make feel like they could be used for the same purpose as Telegram. I still have to check the three later, but please be bold with any feedback and complementary ideas you might have on this topic. Although Mastodon doesn't seem to be what I was looking for at start, I do think it would be great to launch a Wikimedia instance and completely in phase with the aim of becoming an essential infrastructure of the ecosystem of free knowledge. So let me know if I can help in any way on this regard. :) Cheers. [1] https://lists.wikimedia.org/pipermail/wikimedia-l/2018-April/089977.html Le 11/04/2018 à 11:17, Quim Gil a écrit : (These are personal opinions based on my own personal interest in free and volunteer-driven social networks, not an opinion as a WMF member.) On Mon, Apr 9, 2018 at 9:46 AM, Leinonen Teemu wrote: Hi, I have been looking for social networking service that would be fair: not abusing personal data, funded by community, respecting privacy, accepting anonymity, free/libre/ open source etc. Haven’t found many. The Diaspora* Project[1] is not moving forward very fast and the Mastodon[2] is more a microblogging service rather than a social network service. Can it be that the difference between "microblogging service" and "social network" might be too subtle and subjective to be noticed by the majority of their users? And for the problem you are presenting here? Would it make sense for Wikimedia movement to build its own social network service? Depends on what you mean by "build". If you mean create the software for a new social network service, I don't think it makes sense. Providing support and development of multilingual wiki projects <https://wikimediafoundation.org/wiki/Our_projects> to collect and develop educational content to empower and engage people around the world is already a daunting task in terms of software development, and there is so much to do. If you mean to run the software developed by someone else, sure, why not experimenting. Thanks to free software licenses anyone can try, and thanks to Wikimedia trademarks licenses I am sure a decent solution could be found by whoever wants to run this experiment. In the "2017 Movement strategy” we state: “By 2030, Wikimedia will become the essential infrastructure of the ecosystem of free knowledge”. If we consider discussions and information shared on social network services to be “knowledge”, I think we should have a role in here too. With some caveats and observations, I agree on the principle, just not on the implication that this means we need to create a free social network for us from scratch, starting with a first line of code. If we consider social networks useful, and free social networks the right and consistent thing to use in an ecosystem of free knowledge, then the first step can be as simple as opening a Mastodon instance. Dozens (hundreds) of volunteers (including amateur sysadmins) are doing just that without much discussion, just scratching their own itch, or for fun, or to learn, or to experiment... We have 33 million registered users and fulfil all the requirements of being a “fair service”. A minimum
Re: [Wikipl-l] Odp: Co się stało w styczniu
ewidentnie dane pokazują ponad 729k edycji botów w styczniu. Ciekawe. http://u.42.pl/SzZ0 masti On 09.07.2018 15:34, Piotr Gackowski wrote: Tutaj po skróceniu http://tiny.pl/gs36b bo normalnie tam są dziwne znaczki w URLu i nie każdemu dobrze się otworzy. PMG W dniu 9 lipca 2018 15:28 użytkownik Remedios44 napisał: Nie działa mi podany przez ciebie link i nie wiem jak duży to był skok, ale może ma to jakiś związek z tym: pl.wikipedia.org pl.wikipedia.org ewentualnie jakimiś innymi projektami szkolno-akademickimi? Salicyna Dnia 9 lipca 2018 14:56 Piotr Gackowski pmg...@wp.pl napisał(a): Hej Czy ktoś się orientuje co się stało w styczniu 2018 na pl.wiki, ze mamy tak wysoki poziom edycji? stats.wikimedia.org stats.wikimedia.org contributing/edits/normal|bar| PMG __ Lista dyskusyjna WikiPL-l WikiPL-l@lists.wikimedia.org lists.wikimedia.org lists.wikimedia.org ___ Lista dyskusyjna WikiPL-l WikiPL-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikipl-l ___ Lista dyskusyjna WikiPL-l WikiPL-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikipl-l ___ Lista dyskusyjna WikiPL-l WikiPL-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikipl-l
Re: [Wikitech-l] MediaWiki and OpenID Connect
Adam, this is not technical but community problem. That should be first discussed with the communities - all of them - not only en.wiki whether there is any need for that solution. Which I doubt. I do not see many positives coming from using it but a lot of negatives. Enforcing that would mean having much less contributors. Also as already stated it would force us into getting more information about contributors which is not what the communities want. Especially in the moment of upcoming GDPR regulation that affects whole commmunity. masti On 04.05.2018 22:21, Adam Sobieski wrote: Wikitech-l, Greetings. I would like to describe an exciting scenario possible with OpenID Connect. In the scenario, after choosing to verify their name on their Wikipedia account, a user logs onto Wikipedia and uses OpenID Connect to link their Wikipedia account to multiple verified accounts, for example their Facebook and LinkedIn accounts. At the end of the process, we can envision the user obtaining a checkmark next to their full name on Wikipedia, their real name and a verification icon appearing next to their edits and on their user page. There might even be, per user settings, hyperlinks to their Facebook and LinkedIn pages on their Wikipedia user page. With such features, we can envision allowing groups of users or admins to determine that certain articles require a verified account to edit. Presently, OpenID Connect functionality is available for MediaWiki as an extension. I would like to see the OpenID Connect functionality under discussion expanded to support scenarios including aforementioned and also integrated into MediaWiki. Thank you. I hope that the above ideas are also interesting to you in the Wikitech-l community. Best regards, Adam Sobieski http://www.phoster.com/contents/ ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikidata] Citizens of countries
Gerard, you start from this query that I use for ListeriaBot https://pl.wikipedia.org/wiki/Wikipedia:Wikipedia_te%C5%BC_jest_kobiet%C4%85/Brakuj%C4%85ce/szwedzkie_pisarki It lists articles about swedish woman writers that do not have article on polish Wikipedia. Should be easy to modify it to your needs. masti On 16.04.2018 11:19, Gerard Meijssen wrote: Hoi, A friend asked if we have a query showing "humans" sorted by "country of citizenship". He is particularly interested in Mozambique, I am particularly interested in former countries. What would be particularly helpful is when there is a separate column for the genders. For some former countries like the Ottoman Empire it takes more effort to find women to write about.. Making them visible is important. Thanks, GerardM ___ Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata ___ Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Re: [Wikidata] Wikidata and OpenStreetMap
OSM already uses wikidata tag for connetion of the object to WD items. but with the names it is not that easy as Yaroslav said. I do not think we are yet ready to do imports right now changing the placenames. OSM idea is to have the representation of what really exists in the ground. So some names could be different from "official", even as it is also reflected. I am in fact right now investigating synergies for my talk at State-of-the-Map Poland conference in April. http:/sotm-pl.eu masti On 27.03.2018 13:57, Amir E. Aharoni wrote: Hi, *** Before I begin: I've never been a major OpenStreetMap contributor, so forgive me if I misunderstand something basic about it. *** Lately, some work has been done on improving the integration of OpenStreetMap (OSM) and Wikimedia projects, in the Kartographer extension. In particular, I'm curious about this task: https://phabricator.wikimedia.org/T112948 It's about showing place names in the wiki language. It may get resolved soon (yay!!) But it raises an important question: What happens if the place name was not translated into the wiki language? As a not-so-extreme example, what happens if a place name is only available in the OSM database in Chinese? Unfortunately, it will be not very useful to readers of the English Wikipedia. The desirable solution is to give Wikipedia editors who know the relevant languages an easy way to translate the labels. Reading https://wiki.openstreetmap.org/wiki/Translation#OpenStreetMap_website_interface , I see that there is no *easy* way to do it on the OpenStreetMap side. To add a translation of a place name, you need to: * find it on the map * edit it * type "name:LANGUAGE_CODE" in the properties list (for example "name:ru" for Russian) * write the name * save * wait for it to get published (I'm not sure how long does it take; maybe it's instant, but I made a test edit, and I still don't see it.) This is not super-efficient for several reasons: * Finding each place on the map may be time-consuming for practical considerations. * Sending each change manually is also time-consuming. * Typing the property name manually is slowish and error-prone. A lot of this data is already available on Wikidata. In fact, OpenStreetMap already has a Wikidata item property for each place. (It can also have a property for Wikipedia link, one for each language. It looks redundant to me: A link to the Wikidata item page would be enough.) Did anybody ever suggest importing the place names available on Wikidata to OSM, or to synchronize them regularly? -- Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי http://aharoni.wordpress.com “We're living in pieces, I want to live in peace.” – T. Moore ___ Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata ___ Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
[Pywikipedia-bugs] [Maniphest] [Created] T187748: noreference.py should again add section before {{Przypisy}}
Masti created this task.Masti added a project: Pywikibot-core.Herald added subscribers: pywikibot-bugs-list, Aklapper.
TASK DESCRIPTIONAfter the chnage in references template {{Przypisy}} it is necesary to add == Przypisy == section title before templateTASK DETAILhttps://phabricator.wikimedia.org/T187748EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MastiCc: Aklapper, Masti, pywikibot-bugs-list, Magul, Tbscho, rafidaslam, MayS, Mdupont, JJMC89, Avicennasis, jayvdb, Dalba, Alchimista, Rxy___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Created] T180309: reflinks.py changes existing ref names and do not observe ref groups in templates
Masti created this task.Masti added a project: Pywikibot-Other-scripts.Herald added subscribers: pywikibot-bugs-list, Aklapper.Herald added a project: Internet-Archive.
TASK DESCRIPTIONas reflinks.py by default runs renerence deduplication it breaks existing ref groups if templates are used .
in the example below it changes existing group name from to autogenerated
in the result it breaks the connection to ref groups introduced by templates {{r|CIAAW}}{{u|CIAAW}}
{{r}} is connected to grouped References - english equivalent {{r})
{{u}} is connected to grouped Notes
$ python pwb.py reflinks.py -page:Azot -lang:pl
Retrieving 1 pages from wikipedia:pl.
PDF file.
Bad link : http://web.archive.org/web/20140226213530/http://cybra.lodz.pl/Content/291/ChePol+X+12.pdf in [[Azot]]
Azot <<<
@@ -178 +178 @@
* {{IUPAC-SAW2013}}
+ * {{IUPAC-SAW2013}}
Script should not change the name of the reference if it finds one.
Or to introduce a new switch -nodedup to disalow deduplication.
More complex is to observe all reference group introducing templates but that would be to time consuming to implementTASK DETAILhttps://phabricator.wikimedia.org/T180309EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MastiCc: Aklapper, pywikibot-bugs-list, Masti, Jay8g___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Updated] T180308: reflinks.py [SSL: CERTIFICATE_VERIFY_FAILED]
Masti edited projects, added Pywikibot-Other-scripts; removed Internet-Archive.Herald added a subscriber: pywikibot-bugs-list. TASK DETAILhttps://phabricator.wikimedia.org/T180308EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MastiCc: pywikibot-bugs-list, Masti, Aklapper___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Closed] T175811: logentries.py fails on call to user()
Masti closed this task as "Invalid".Masti added a comment. I have found the real reason for the error which was a not catched wrong entry in the delete log. That makes the task invalidTASK DETAILhttps://phabricator.wikimedia.org/T175811EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MastiCc: Aklapper, pywikibot-bugs-list, Masti, Magul, Tbscho, MayS, Mdupont, JJMC89, Avicennasis, jayvdb, Dalba, Alchimista, Rxy___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Created] T175811: logentries.py fails on call to user()
Masti created this task.Masti added projects: Pywikibot-core, Pywikibot-Other-scripts.Herald added subscribers: pywikibot-bugs-list, Aklapper.
TASK DESCRIPTIONWhile reading logentry via le.user() (le is an logentry type object) it fails with the error KeyError: u"Log entry (delete) has no 'user' key"
ms-deletestats.py script analyses logentries for deletion actions
masti@tools ~/pw/core $ time python pwb.py masti/ms-deletestats.py -page:'!' -outpage:'deletes.html'
Traceback (most recent call last):
File "pwb.py", line 262, in
if not main():
File "pwb.py", line 255, in main
run_python_file(filename, [filename] + args, argvu, file_package)
File "pwb.py", line 121, in run_python_file
main_mod.__dict__)
File "./masti/ms-deletestats.py", line 443, in
main()
File "./masti/ms-deletestats.py", line 436, in main
bot.run() # guess what it does
File "./masti/ms-deletestats.py", line 148, in run
deletereasons = self.deleteReasons(currtime,int(self.getOption('days')))
File "./masti/ms-deletestats.py", line 172, in deleteReasons
pywikibot.output(u'%i>>%s>>%s>>%s>>%s>>%s>>%s' % (count, le.type(), le.logid(),le.timestamp(),le.action(),le.user(),comment))
File "/home/mst/pw/core/pywikibot/logentries.py", line 139, in user
return self.data['user']
File "/home/mst/pw/core/pywikibot/logentries.py", line 37, in __missing__
raise KeyError("Log entry (%s) has no '%s' key" % (self._type, key))
KeyError: u"Log entry (delete) has no 'user' key"
CRITICAL: Closing network session.
python pwb.py version.py
Pywikibot: [https] r-pywikibot-core (22d19cf, g8626, 2017/09/12, 13:50:19, n/a)
Release version: 3.0-dev
requests version: 2.12.5
cacerts: /etc/ssl/certs/ca-certificates.crt
certificate test: ok
Python: 2.7.12 (default, Apr 24 2017, 01:43:29)
[GCC 4.9.3]
PYWIKIBOT2_DIR: Not set
PYWIKIBOT2_DIR_PWB:
PYWIKIBOT2_NO_USER_CONFIG: Not set
Config base dir: /usr/home/masti/pw/core
Usernames for family "wikipedia":TASK DETAILhttps://phabricator.wikimedia.org/T175811EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MastiCc: Aklapper, pywikibot-bugs-list, Masti, Magul, Tbscho, MayS, Mdupont, JJMC89, Avicennasis, jayvdb, Dalba, Alchimista, Rxy___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T55936: reflinks.py work with ref group
Masti added a comment. one option could be to add -nogroup or simmilar option to skip any changes to existing reference groups and not to goup existing refs. This is a kind of cosmetic changes to refs and the bot shoul d be able to skip it.TASK DETAILhttps://phabricator.wikimedia.org/T55936EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MastiCc: Masti, Aklapper, Legoktm, jayvdb, pywikibot-bugs-list, Rubin16, Abiyoyo, Magul, Tbscho, MayS, Jack_who_built_the_house, Mdupont, JJMC89, Iniquity, Avicennasis, Anshoe, Sn1per, Dalba, stjn, putnik, MaxBioHazard, Alchimista, Rxy___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T114807: Editing as sysop doesn't work
Masti added a comment. patch 341699 submitted https://gerrit.wikimedia.org/r/#/c/341699/TASK DETAILhttps://phabricator.wikimedia.org/T114807EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MastiCc: Masti, Edgars2007, JAnD, Aklapper, Alkamid, pywikibot-bugs-list, Magul, Tbscho, MayS, Mdupont, JJMC89, Avicennasis, Sn1per, jayvdb, Dalba, Alchimista, Rxy___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T119335: as_group='sysop' not working properly
Masti added a comment. using BotPasswords also does not help.TASK DETAILhttps://phabricator.wikimedia.org/T119335EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MastiCc: Masti, Aklapper, StudiesWorld, Alkamid, pywikibot-bugs-list, Magul, Tbscho, MayS, Mdupont, JJMC89, Avicennasis, Sn1per, jayvdb, Dalba, Alchimista, Rxy___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Retitled] T151733: Compat deprecation message broke compat
Masti changed the title from "Compat deprecation message broke comat" to "Compat deprecation message broke compat". TASK DETAILhttps://phabricator.wikimedia.org/T151733EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MastiCc: Aklapper, Masti, pywikibot-bugs-list, Anshoe___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Created] T151733: Compat deprecation message broke comat
Masti created this task.Masti added a project: Pywikibot-compat.Herald added subscribers: pywikibot-bugs-list, Aklapper.
TASK DESCRIPTIONwarning message in throttle.py introduced in https://gerrit.wikimedia.org/r/322502 crasshes with
EOFError: EOF when reading a line
example:
Press to continue.
Traceback (most recent call last):
File "basicvotes.py", line 29, in
import wikipedia as pywikibot
File "/usr/home/masti/pw/compat/wikipedia.py", line 10466, in
get_throttle = Throttle()
File "/usr/home/masti/pw/compat/pywikibot/throttle.py", line 66, in __init__
self.checkMultiplicity()
File "/usr/home/masti/pw/compat/pywikibot/throttle.py", line 140, in checkMultiplicity
self._show_warning()
File "/usr/home/masti/pw/compat/pywikibot/throttle.py", line 89, in _show_warning
\03{default}""")
File "/usr/home/masti/pw/compat/wikipedia.py", line 10329, in input
data = "" password)
File "/usr/home/masti/pw/compat/userinterfaces/terminal_interface_base.py", line 142, in input
text = self._raw_input()
File "/usr/home/masti/pw/compat/userinterfaces/terminal_interface_base.py", line 120, in _raw_input
return raw_input()
EOFError: EOF when reading a lineTASK DETAILhttps://phabricator.wikimedia.org/T151733EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MastiCc: Aklapper, Masti, pywikibot-bugs-list, Anshoe, jayvdb___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Created] T150694: selflink.py - AttributeError: 'SelflinkBot' object has no attribute 'pageToUnlink'
Masti created this task.Masti added a project: Pywikibot-core.Herald added subscribers: pywikibot-bugs-list, Aklapper. TASK DESCRIPTIONselflink.py halts while calling unlink.py with the following error: $ python pwb.py selflink.py -start:A Retrieving 50 pages from wikipedia:pl. A <<< 0 pages read 0 pages written Execution time: 0 seconds Script terminated by exception: ERROR: AttributeError: 'SelflinkBot' object has no attribute 'pageToUnlink' Traceback (most recent call last): File "pwb.py", line 255, in if not main(): File "pwb.py", line 249, in main run_python_file(filename, [filename] + args, argvu, file_package) File "pwb.py", line 121, in run_python_file main_mod.__dict__) File "./scripts/selflink.py", line 110, in main() File "./scripts/selflink.py", line 106, in main bot.run() File "/usr/home/masti/pw/core/pywikibot/bot.py", line 1621, in run super(MultipleSitesBot, self).run() File "/usr/home/masti/pw/core/pywikibot/bot.py", line 1435, in run self.treat(page) File "/usr/home/masti/pw/core/pywikibot/bot.py", line 1722, in treat super(ExistingPageBot, self).treat(page) File "/usr/home/masti/pw/core/pywikibot/bot.py", line 1786, in treat super(NoRedirectPageBot, self).treat(page) File "/usr/home/masti/pw/core/pywikibot/bot.py", line 1649, in treat self.treat_page() File "./scripts/selflink.py", line 74, in treat_page self.unlink(self.current_page) File "/usr/home/masti/pw/core/scripts/unlink.py", line 97, in unlink unlink_callback = self._create_callback() File "./scripts/selflink.py", line 60, in _create_callback callback = super(SelflinkBot, self)._create_callback() File "/usr/home/masti/pw/core/scripts/unlink.py", line 91, in _create_callback return InteractiveUnlink(self) File "/usr/home/masti/pw/core/scripts/unlink.py", line 59, in __init__ old_link=bot.pageToUnlink, new_link=False, default='u') AttributeError: 'SelflinkBot' object has no attribute 'pageToUnlink' CRITICAL: Closing network session. run on a clean new core installation $ python pwb.py version.py Pywikibot: [https] r-pywikibot-core.git (1b3d330, g7586, 2016/11/14, 14:21:21, n/a) Release version: 3.0-dev requests version: 2.3.0 cacerts: /etc/ssl/certs/ca-certificates.crt certificate test: ok Python: 2.7.7 (default, Dec 9 2014, 00:19:44) [GCC 4.3.4] PYWIKIBOT2_DIR: Not set PYWIKIBOT2_DIR_PWB: PYWIKIBOT2_NO_USER_CONFIG: Not setTASK DETAILhttps://phabricator.wikimedia.org/T150694EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MastiCc: Aklapper, pywikibot-bugs-list, Masti, MayS, Mdupont, JJMC89, jayvdb, Dalba, Alchimista, Rxy___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
Re: [pywikibot] Not sure if I'm running weblinkchecker correctly
look for the subdirectory called
deadlinks
it contains all data generated by the bot
results-yourwiki.txt - a bot log (text)
deadlinks-yourwiki.dat - a bot database (dump of python
dictionary)
so You can see what is going on
masti
On 05/28/2016 11:36 PM, . wrote:
Ok. I had added some dead links to the user page
and was testing to see what it would do. Thank you!
On Sat, May 28, 2016 at 4:02 PM, masti
<mast...@gmail.com>
wrote:
Hi,
all the examples are in the script help.
You have to run it on a list of pages not just Your user
page
the bot will create a database of the links it thinks
are dead.
the bot will only report links as dead if they are found
dead at least twice several days apart. Default is 7
days.
So once You run it through the whole namespace You will
have to wait some days to rerun it.
You can set the rquired period with -day parameter
but less than a week does not make sense as it will
report dead link for a temporary problems with site
masti
On 05/28/2016 07:18 PM, . wrote:
Hi all,
I'my trying to run weblinkchecker from my own
computer. I've logged in to the bot account, but
when I try to run weblink checker, like this:
>python weblinkchecker.py
UserPage:usernamegoeshere
I just get a very long bunch of help text on
the screen "This bot is used for checking
external links found at the wiki" etc. I don't
know if I'm successfully running the script or
not, and whether it's actually checking any
links.
What am I doing wrong?
Thanks!
___
pywikibot mailing list
pywikibot@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot
___
pywikibot mailing list
pywikibot@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot
___
pywikibot mailing list
pywikibot@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot
___
pywikibot mailing list
pywikibot@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot
Re: [pywikibot] Not sure if I'm running weblinkchecker correctly
Hi, all the examples are in the script help. You have to run it on a list of pages not just Your user page the bot will create a database of the links it thinks are dead. the bot will only report links as dead if they are found dead at least twice several days apart. Default is 7 days. So once You run it through the whole namespace You will have to wait some days to rerun it. You can set the rquired period with -day parameter but less than a week does not make sense as it will report dead link for a temporary problems with site masti On 05/28/2016 07:18 PM, . wrote: Hi all, I'my trying to run weblinkchecker from my own computer. I've logged in to the bot account, but when I try to run weblink checker, like this: >python weblinkchecker.py UserPage:usernamegoeshere I just get a very long bunch of help text on the screen "This bot is used for checking external links found at the wiki" etc. I don't know if I'm successfully running the script or not, and whether it's actually checking any links. What am I doing wrong? Thanks! ___ pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot ___ pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
[Wikipl-l] Fwd: [Wikimania-l] Wikimania 2015 documentary teaser
___ Wikimania-l mailing list wikimani...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikimania-l ___ Lista dyskusyjna WikiPL-l WikiPL-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikipl-l
Re: [treasurers] Budget for Wikimania 2016 scholarships
On 11/10/2015 08:08 AM, Federico Leva (Nemo) wrote: Dear treasurers, I'm collecting some public information on Wikimania scholarships budgets at https://meta.wikimedia.org/wiki/Wikimania/Scholarships/2016 (the table is already interesting). If you already have some decision, please add details about your entity; if you don't, please start discussing in your board immediately! See timeline: https://wikimania2016.wikimedia.org/wiki/Scholarship_committee Federico Wikimania 2016 team, scholarships subteam ___ Treasurers mailing list Treasurers@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/treasurers Frederico, do You have any idea what will be the approximate cost? like Attendance fee, accomodation etc? Marek masti Stelmasik Wikimedia Polska ___ Treasurers mailing list Treasurers@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/treasurers
Re: [pywikibot] weblinchecker data migration from compat to core
On 08/07/2015 11:50 PM, Merlijn van Deen wrote: Hi masti, On 7 August 2015 at 23:11, masti mast...@gmail.com mailto:mast...@gmail.com wrote: weblinchecker.py working file deadlinks-wikipedia-pl.dat has a binary format in core. In Compat it was a text file. This confuses me. As far as I can see, both compat and core use 'pickle' to write the working file (and as far as I can see, compat always has). There are, however, different versions of the pickle format, which could explain the binary/text difference. Could you post an excerpt of the old file? (dp0 Vhttp://www.european-athletics.org/european-athletics-awards-night/baldini-says-goodbye-at-the-giro-al-sas.html p1 (lp2 (VStefano Baldini p3 F1404150662.121833 S'404 Not Found' p4 tp5 a(V2010 w lekkoatletyce p6 F1404569856.34401 S'404 Not Found' p7 tp8 a(VStefano Baldini p9 F1404729661.197063 S'404 Not Found' p10 tp11 a(V2010 w lekkoatletyce p12 F1404761790.987255 S'404 Not Found' p13 tp14 a(VStefano Baldini p15 F1413308613.299339 S'404 Not Found' p16 tp17 the script works after copying it to the new directory but it looks it does not recognize information about previously reported links. Is there an error message posted, is the file just being overwritten, or is there some other kind of issue? no. no error, it does not overwrite the file. I am still testing wether it adds the records. But it tries to report deadlinks the were previously reported. So this makes me think the script does not recognize properly old records. Best regards, Merlijn ___ pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot ___ pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
[pywikibot] weblinchecker data migration from compat to core
weblinchecker.py working file deadlinks-wikipedia-pl.dat has a binary format in core. In Compat it was a text file. the script works after copying it to the new directory but it looks it does not recognize information about previously reported links. As I ran it since some years on pl.wiki I need to preserve this info to not to report the links. is there a way to migrate the data to the new format or a descritpion of both formats so I can do a file conversion? masti ___ pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
Re: [Translators-l] 3 minutes of your time: a video about editathons
done for polish masti On 06/04/2015 12:24 AM, Amir E. Aharoni wrote: Hi dear translators, I'm passing on a request from my friend and colleague Victor Grigas. There's this short video about editathons: https://commons.wikimedia.org/wiki/File:WikipediaMeetup.webm It is going to be distributed on Friday. It would be very nice if you could get it translated to your language. I just did it for Hebrew and it took me less than 3 minutes. Instructions: 1. Copy the text from https://commons.wikimedia.org/wiki/TimedText:WikipediaMeetup.webm.en.srt 2. Create a page similar to the URL above, but with your language code instead of en. For example, for French write: https://commons.wikimedia.org/wiki/TimedText:WikipediaMeetup.webm.fr.srt 3. Paste the English text that you copied in step 1. 4. Change the English lines to lines in your language. Don't change the lines with numbers and times. 5. Save the page. That's it! To test, go again to https://commons.wikimedia.org/wiki/File:WikipediaMeetup.webm ... and while playing the video, click the CC button at the bottom of the player and select your language. Thank you! -- Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי http://aharoni.wordpress.com “We're living in pieces, I want to live in peace.” – T. Moore ___ Translators-l mailing list Translators-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/translators-l ___ Translators-l mailing list Translators-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/translators-l
Re: [Wikipl-l] Kierownik prasowy Stowarzyszenia Wikimedia Polska zmusi cię do współpracy, nawet gdy nie odpowiadają ci warunki.
Tomku, przepraszam ale mylisz edytorów ze społecznością. Pomijam już boty i stewardół ale to, żę ktoś wykonał edycję nie oznacza, że czuje się i jest członkiem społeczności w sensie poczucia przynależności i współdecydowania o projekcie. masti On 01/02/2015 07:46 PM, Tommy Jantarek wrote: oto obecna lista przypadkowych userów, większość nie bierze udziału w dyskusji, wg Awersowego nie są społecznością https://pl.wikisource.org/wiki/Specjalna:Aktywni_u%C5%BCytkownicy ___ Lista dyskusyjna WikiPL-l WikiPL-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikipl-l ___ Lista dyskusyjna WikiPL-l WikiPL-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikipl-l
l4linux shutdown cleanup
Dear all, I would like to perform some cleanup operations after l4linux shuts down..such as zero out all the memory it used and re-encrypt the original vmlinux image. I would like to know how I could do this from sigma0 preferably or even in the fiasco kernel. Could anyone point me to where the clean up procedure (release of resources) occurs once l4linux shuts down? Thanks, Ramya ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
Ned,moe, l4linux interaction
Dear all, I see that in the default l4linux configuration, it uses ned, moe to start up. However, I am not sure about the run-time interaction between them. More specifically, a. Does l4linux need them during its run-time? For example, does l4linux go directly to sigma0 for memory or does it go through ned? b. Do ned and moe get killed once they start l4linux or not? Thanks, ramya ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
RE: Trimming uclibc in l4
Hi, I completely understand that I should update but given time constraints, I am using an older version on a board that it works. I will update it eventually. I managed to backport the library which was 2 revisions ahead. In this version only the bootstrap module uses the minimal lib. I removed some more files from the minimal library (which my modified version does not need) and inspected the build folder. The actual library reduces in size expected. However, I see that the bootstrap.elf executable increases in size. Furthermore, when I use the make elfimage command (e.g., make elfimage E=l4linux), changing the list of files in the uclibc library does not seem to have any effect on the final executable's size (i.e., bootstrap_l4linux.elf). I can confirm that bootstrap builds against the minimal library because not having it there simply cause the build to fail. However, I do not understand why modifying the list of files in the library does not change anything. Thanks in advance, ramya From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Björn Döbel [doe...@os.inf.tu-dresden.de] Sent: 16 September 2014 19:15 To: l4-hackers@os.inf.tu-dresden.de Subject: Re: Trimming uclibc in l4 Hi, as Marcus already pointed out, newer versions of L4Re actually contain a package called uclibc_minimal that serves exactly this purpose. Even if you are using an older snapshot, chances are that backporting this package (and adapting l4/mk) might be easier than stripping things down yourself. Bjoern Am 16.09.2014 um 19:02 schrieb Masti Ramya Jayaram: Update: turns out it is not very simple to compile just bootstrap against a smaller library because the l4/mk/modes.inc file defaults to the uclibc library for compiling the files which is used for all packages. Any ideas how I can change this only for some packages? Thanks. Ramya From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Masti Ramya Jayaram [rma...@inf.ethz.ch] Sent: 16 September 2014 17:53 To: l4-hackers@os.inf.tu-dresden.de Subject: Trimming uclibc in l4 Dear all, I would like to extract the minimal subset of uclibc required by bootstrap, sigma and their dependencies only. My idea is to have these modules use the smaller libc while retain the usual uclibc build for the other packages. I intend to go about this by copying uclibc (say into uclibc_min) and then modifying the contrib_files_all.lst and contrib_files_x86.lst. Is this the correct way to do this? Also, I extracted all the package dependencies of bootstrap and sigma from the control files (below) bootstrap: drivers_uart drivers_of libc l4util cxx_io drivers-frst:libc l4util: crtn libc crtn: l4sys l4sys: ldscripts l4util: crtn libc cxx: l4sys l4util ulibc: l4sys sigma0: crtn l4sys l4util libsigma0 cxx_io libsigma0: l4sys Is this the correct way to do this as well? Thanks, Ramya ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
Re: Physical memory allocation to L4linux
Hey Adam, Yes I can confirm that I was a able to trace the code from fpage_map() to map() but I see zero addresses there - i.e. When I check the receiver and sender addresses and size everything is zero. This was despite it being correct until the handle_sigma0_request() function in mem_map.cc. So I just assumed I was looking in the wrong place. I will try to dig a little more and check. Sent from my phone On 15 Sep 2014, at 23:55, Adam Lackorzynski a...@os.inf.tu-dresden.de wrote: Hi, On Mon Sep 15, 2014 at 18:25:37 +, Masti Ramya Jayaram wrote: Thanks for the input. I will try changing one of the suggested functions. It would be great out of curiosity if you could elucidate why the following occurs. I see that there is a new iomem region being created and this happens only when I try to read from the mapped region - not until then. But none of this reaches the kernel or maybe I am not sure where to look.. Well, it should reach the kernel. For sigma0 mappings appear out of nothing (it's the root), so you should at least see it in the mapping loop in map_util.cpp's map() function (also see the comments on sigma0 there). In fact, I see that the init_mem file parses the kernel memory descriptors to find the regions. So under what circumstances does sigma0 go back to the kernel for mem/io? In this case there a fpage is created that is used in the reply to the client which will then create the mapping. Adam -- Adam a...@os.inf.tu-dresden.de Lackorzynski http://os.inf.tu-dresden.de/~adam/ ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
Trimming uclibc in l4
Dear all, I would like to extract the minimal subset of uclibc required by bootstrap, sigma and their dependencies only. My idea is to have these modules use the smaller libc while retain the usual uclibc build for the other packages. I intend to go about this by copying uclibc (say into uclibc_min) and then modifying the contrib_files_all.lst and contrib_files_x86.lst. Is this the correct way to do this? Also, I extracted all the package dependencies of bootstrap and sigma from the control files (below) bootstrap: drivers_uart drivers_of libc l4util cxx_io drivers-frst:libc l4util: crtn libc crtn: l4sys l4sys: ldscripts l4util: crtn libc cxx: l4sys l4util ulibc: l4sys sigma0: crtn l4sys l4util libsigma0 cxx_io libsigma0: l4sys Is this the correct way to do this as well? Thanks, Ramya ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
RE: Trimming uclibc in l4
Update: turns out it is not very simple to compile just bootstrap against a smaller library because the l4/mk/modes.inc file defaults to the uclibc library for compiling the files which is used for all packages. Any ideas how I can change this only for some packages? Thanks. Ramya From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Masti Ramya Jayaram [rma...@inf.ethz.ch] Sent: 16 September 2014 17:53 To: l4-hackers@os.inf.tu-dresden.de Subject: Trimming uclibc in l4 Dear all, I would like to extract the minimal subset of uclibc required by bootstrap, sigma and their dependencies only. My idea is to have these modules use the smaller libc while retain the usual uclibc build for the other packages. I intend to go about this by copying uclibc (say into uclibc_min) and then modifying the contrib_files_all.lst and contrib_files_x86.lst. Is this the correct way to do this? Also, I extracted all the package dependencies of bootstrap and sigma from the control files (below) bootstrap: drivers_uart drivers_of libc l4util cxx_io drivers-frst:libc l4util: crtn libc crtn: l4sys l4sys: ldscripts l4util: crtn libc cxx: l4sys l4util ulibc: l4sys sigma0: crtn l4sys l4util libsigma0 cxx_io libsigma0: l4sys Is this the correct way to do this as well? Thanks, Ramya ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
RE: Trimming uclibc in l4
Hey Markus, I am using a version from April 2011 but even that version contains a lot of things that bootstrap and sigma0 do not need. :) Did you mean more recent than that? best, ramya p.s. my board requires that I run an older version of the kernel - so the pain. From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Marcus Hähnel [mhaeh...@os.inf.tu-dresden.de] Sent: 16 September 2014 18:31 To: l4-hackers@os.inf.tu-dresden.de Subject: Re: Trimming uclibc in l4 Hi, I would like to extract the minimal subset of uclibc required by bootstrap, sigma and their dependencies only. My idea is to have these modules use the smaller libc while retain the usual uclibc build for the other packages. Great idea, but you should probably update your L4Re. L4Re features such a minimal version since about 2011. So no need to do the same work twice :) - Marcus I intend to go about this by copying uclibc (say into uclibc_min) and then modifying the contrib_files_all.lst and contrib_files_x86.lst. Is this the correct way to do this? Also, I extracted all the package dependencies of bootstrap and sigma from the control files (below) bootstrap: drivers_uart drivers_of libc l4util cxx_io drivers-frst:libc l4util: crtn libc crtn: l4sys l4sys: ldscripts l4util: crtn libc cxx: l4sys l4util ulibc: l4sys sigma0: crtn l4sys l4util libsigma0 cxx_io libsigma0: l4sys Is this the correct way to do this as well? Thanks, Ramya ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
Re: Physical memory allocation to L4linux
Hey Adam, Thanks for the input. I will try changing one of the suggested functions. It would be great out of curiosity if you could elucidate why the following occurs. I see that there is a new iomem region being created and this happens only when I try to read from the mapped region - not until then. But none of this reaches the kernel or maybe I am not sure where to look.. In fact, I see that the init_mem file parses the kernel memory descriptors to find the regions. So under what circumstances does sigma0 go back to the kernel for mem/io? Thanks a lot! Ramya On 15 Sep 2014, at 00:38, Adam Lackorzynski a...@os.inf.tu-dresden.de wrote: On Fri Sep 12, 2014 at 16:34:17 +, Masti Ramya Jayaram wrote: I realized that the best place to modify sigma0's capabilities are when it is created. I see that this happens in kernel/fiasco/src/kern/kernel_thread-std.cpp in the init_workload() function. I do not exactly understand how sigma0 gathers all the capabilities - the code is a little hard to parse for me. Could someone explain this to me? Is there a way to pass only limited capabilities to sigma0 -aka, allow the default and then prune the resource tree? So, I've changed my mind, let's not modify the kernel but sigma0. This seems easier to me. As sigma0 is the root of memory this should not be a problem and can be argued. Sigma0 handles memory at first-come-first-serve approach and only gives it out once, so allocating it before servicing client requests should hide it from clients (see memmap.cc). However, there's also an unmap call which would need to be handled to not free it again. Another approach could be to exclude the memory region when initializing the internal data structures in init_mem.cc. Whatever works/fits better. Adam -- Adam a...@os.inf.tu-dresden.de Lackorzynski http://os.inf.tu-dresden.de/~adam/ ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
RE: Physical memory allocation to L4linux
Hi,
I realized that the best place to modify sigma0's capabilities are when it is
created. I see that this happens in
kernel/fiasco/src/kern/kernel_thread-std.cpp in the init_workload() function. I
do not exactly understand how sigma0 gathers all the capabilities - the code is
a little hard to parse for me. Could someone explain this to me? Is there a way
to pass only limited capabilities to sigma0 -aka, allow the default and then
prune the resource tree?
For now, my sigma0 output report that it has the following:
SIGMA0: Dump of all resource maps
RAM:
[0:10b000;13]
[4:14;18cfff]
[0:18d000;2ecfff]
[4:2ed000;2edfff]
[0:2ee000;3f]
[0:45b000;119cfff]
[4:119d000;1360fff]
[0:1361000;25b98fff]
IOMEM:--
[0:0;f]
[0:2900;fedf]
[0:fee01000;]
IO PORTS--
[0:0;fff]
is there a way for me for example to limit RAM and IOMEM access to other
spaces?
Thanks,
ramya
From: Masti Ramya Jayaram
Sent: 11 September 2014 15:51
To: Adam Lackorzynski; l4-hackers@os.inf.tu-dresden.de
Subject: RE: Physical memory allocation to L4linux
Hey guys,
I see that the IO request reaches the io_map function in
kernel/fiasco/src/kern/ia32/map_util-io.cpp:
L4_error
io_map(Space *from, L4_fpage const fp_from,
Space *to, L4_fpage const fp_to, Mword control)
{
typedef Map_traitsIo_space Mt;
Mt::Addr rcv_pos = Mt::get_addr(fp_to);
Mt::Addr snd_pos = Mt::get_addr(fp_from);
Mt::Size rcv_size = Mt::Size::from_shift(fp_to.order());
Mt::Size snd_size = Mt::Size::from_shift(fp_from.order());
snd_pos = snd_pos.trunc(snd_size);
rcv_pos = rcv_pos.trunc(rcv_size);
Mt::constraint(snd_pos, snd_size, rcv_pos, rcv_size, Mt::Addr(0));
...
}
However, when I do a return L4_error:Map_failed from the beginning of this
function to deny all IO requests, it does not propagate back to sigma0. Could
someone explain how exactly this communication/IPC between the kernel and sigma
works?
Thanks,
Ramya
From: Masti Ramya Jayaram
Sent: 10 September 2014 16:07
To: Adam Lackorzynski; l4-hackers@os.inf.tu-dresden.de
Subject: RE: Physical memory allocation to L4linux
Update: Sorry, I misunderstood a few things in the previous email. The change
was in fiasco and not sigma and therefore, in theory, no user space process
should have access to the protected region. This also implies that sigma0 will
not have access if I understand correctly which is rather nice.
So, I tried changing the handle_sigma0_page_fault function. This is located in
kernel/fiasco/src/kern/ia32/thread-ia32.cpp to prevent mapping of some range of
physical memory (NOT RAM) as follows:
IMPLEMENT inline
bool
Thread::handle_sigma0_page_fault(Address pfa)
{
size_t size;
// Check if mapping a superpage doesn't exceed the size of physical memory
if (Cpu::have_superpages()
// Some distributions do not allow to mmap below a certain threshold
// (like 64k on Ubuntu 8.04) so we cannot map a superpage at 0 if
// we're Fiasco-UX
(!Config::Is_ux || !(pfa Config::SUPERPAGE_SIZE)))
{
pfa = Config::SUPERPAGE_MASK;
size = Config::SUPERPAGE_SIZE;
}
else
{
pfa = Config::PAGE_MASK;
size = Config::PAGE_SIZE;
}
--
- My change start
if((pfa=0xe000) (pfa=0xefff)){
printf(\nNot allowed...denying\n);
return Mem_space::Insert_err_nomem;
}
--
- My change end
return mem_space()-v_insert(Mem_space::Phys_addr(pfa), Mem_space::Addr(pfa),
Mem_space::Size(size),
Mem_space::Page_writable
| Mem_space::Page_user_accessible)
!= Mem_space::Insert_err_nomem;
}
However, I realized that when I try to map this address space using
l4io_request_iomem, I see that it goes through and the call never reaches until
fiasco's sigma0_page_fault handler. A little more debugging revealed that it
goes as far as the following function in l4/pkg/sigma0/server/src/memmap.cc:
static
void map_mem(l4_fpage_t fp, Memory_type fn, l4_umword_t t, Answer *an)
{
an-clear();
Mem_man *m;
switch (fn)
{
case Ram:
m = Mem_man::ram();
break;
case Io_mem:
case Io_mem_cached:
m = iomem;
break;
default:
return;
}
unsigned long addr = m-alloc(Region::bs(fp.raw ~((1UL 12) - 1),
1UL l4_fpage_size(fp), t));
if (addr == ~0UL)
return;
/* the Fiasco kernel makes the page non-cachable if the frame
* address is greater than mem_high */
an-snd_base(addr);
an-snd_fpage(addr
RE: Physical memory allocation to L4linux
Hey guys,
I see that the IO request reaches the io_map function in
kernel/fiasco/src/kern/ia32/map_util-io.cpp:
L4_error
io_map(Space *from, L4_fpage const fp_from,
Space *to, L4_fpage const fp_to, Mword control)
{
typedef Map_traitsIo_space Mt;
Mt::Addr rcv_pos = Mt::get_addr(fp_to);
Mt::Addr snd_pos = Mt::get_addr(fp_from);
Mt::Size rcv_size = Mt::Size::from_shift(fp_to.order());
Mt::Size snd_size = Mt::Size::from_shift(fp_from.order());
snd_pos = snd_pos.trunc(snd_size);
rcv_pos = rcv_pos.trunc(rcv_size);
Mt::constraint(snd_pos, snd_size, rcv_pos, rcv_size, Mt::Addr(0));
...
}
However, when I do a return L4_error:Map_failed from the beginning of this
function to deny all IO requests, it does not propagate back to sigma0. Could
someone explain how exactly this communication/IPC between the kernel and sigma
works?
Thanks,
Ramya
From: Masti Ramya Jayaram
Sent: 10 September 2014 16:07
To: Adam Lackorzynski; l4-hackers@os.inf.tu-dresden.de
Subject: RE: Physical memory allocation to L4linux
Update: Sorry, I misunderstood a few things in the previous email. The change
was in fiasco and not sigma and therefore, in theory, no user space process
should have access to the protected region. This also implies that sigma0 will
not have access if I understand correctly which is rather nice.
So, I tried changing the handle_sigma0_page_fault function. This is located in
kernel/fiasco/src/kern/ia32/thread-ia32.cpp to prevent mapping of some range of
physical memory (NOT RAM) as follows:
IMPLEMENT inline
bool
Thread::handle_sigma0_page_fault(Address pfa)
{
size_t size;
// Check if mapping a superpage doesn't exceed the size of physical memory
if (Cpu::have_superpages()
// Some distributions do not allow to mmap below a certain threshold
// (like 64k on Ubuntu 8.04) so we cannot map a superpage at 0 if
// we're Fiasco-UX
(!Config::Is_ux || !(pfa Config::SUPERPAGE_SIZE)))
{
pfa = Config::SUPERPAGE_MASK;
size = Config::SUPERPAGE_SIZE;
}
else
{
pfa = Config::PAGE_MASK;
size = Config::PAGE_SIZE;
}
--
- My change start
if((pfa=0xe000) (pfa=0xefff)){
printf(\nNot allowed...denying\n);
return Mem_space::Insert_err_nomem;
}
--
- My change end
return mem_space()-v_insert(Mem_space::Phys_addr(pfa), Mem_space::Addr(pfa),
Mem_space::Size(size),
Mem_space::Page_writable
| Mem_space::Page_user_accessible)
!= Mem_space::Insert_err_nomem;
}
However, I realized that when I try to map this address space using
l4io_request_iomem, I see that it goes through and the call never reaches until
fiasco's sigma0_page_fault handler. A little more debugging revealed that it
goes as far as the following function in l4/pkg/sigma0/server/src/memmap.cc:
static
void map_mem(l4_fpage_t fp, Memory_type fn, l4_umword_t t, Answer *an)
{
an-clear();
Mem_man *m;
switch (fn)
{
case Ram:
m = Mem_man::ram();
break;
case Io_mem:
case Io_mem_cached:
m = iomem;
break;
default:
return;
}
unsigned long addr = m-alloc(Region::bs(fp.raw ~((1UL 12) - 1),
1UL l4_fpage_size(fp), t));
if (addr == ~0UL)
return;
/* the Fiasco kernel makes the page non-cachable if the frame
* address is greater than mem_high */
an-snd_base(addr);
an-snd_fpage(addr, l4_fpage_size(fp), false, fn != Io_mem);
an-tag = l4_msgtag(0, 0, 1, 0);
return;
}
From here on, the generics make it a little hard to parse. So I am not sure
where the control flow ends but it does not get to fiasco's sigma0 page fault
handler. Given this,
a. does the handle_sigma0_page_fault only work on memory. The region I want to
protect is not RAM.
b. Given that the setup of sigma0's memory happens in
kernel/fiasco/src/kern/kernel_thread-std.cpp, is there more to be done here?
Thanks.
Ramya
From: Masti Ramya Jayaram
Sent: 10 September 2014 09:46
To: Adam Lackorzynski; l4-hackers@os.inf.tu-dresden.de
Subject: RE: Physical memory allocation to L4linux
Hey Adam,
thanks a lot for the replies. I still have a few questions/clarifications
though. :-)
a. Is there a way to implement the MODE=sigma for linux? I could not find
anything that points to it.
b. Now if I were to tweak the page fault handler in sigma, then does that imply
that even a corrupt moe/ned/l4linux cannot get access to the protected part of
the address space?
c. Who are the clients of this page fault handler? Is it just what runs on top
of sigma like moe, ned, l4linux
RE: Physical memory allocation to L4linux
Hey Adam, thanks a lot for the replies. I still have a few questions/clarifications though. :-) a. Is there a way to implement the MODE=sigma for linux? I could not find anything that points to it. b. Now if I were to tweak the page fault handler in sigma, then does that imply that even a corrupt moe/ned/l4linux cannot get access to the protected part of the address space? c. Who are the clients of this page fault handler? Is it just what runs on top of sigma like moe, ned, l4linux or even faisco? d. If it is just moe, ned, l4linux, then can I use the same page fault trick to ensure that (moe,ned,sigma) they always get memory from a certain dedicated region? Thanks a ton for your patience (to everyone on the list and particularly you and Martin). I totally appreciate it. Best, Ramya From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Adam Lackorzynski [a...@os.inf.tu-dresden.de] Sent: 09 September 2014 23:16 To: l4-hackers@os.inf.tu-dresden.de Subject: Re: Physical memory allocation to L4linux On Tue Sep 09, 2014 at 16:03:11 +, Masti Ramya Jayaram wrote: I would like to explain my situation better. Here is what I intend: a. What is the lowest module (bootstrap, fiasco, sigma, moe, ned,l4linux) that can be confined not access a portion of the address space? Without changing anything and with this list of program, it's just L4Linux. Reason is that ned typically has a cap to sigma0 because it needs to give it to io. When you would like to make a little change, in Fiasco there's a function handle_sigma0_page_fault that covers page-fault by sigma0. So if you add a check on pfa there and return false you should be able to exclude a memory region from any user program. I know that bootstrap and fiasco run in privileged mode, so there is no way to stop them. What is the next module? b. Assuming that it is sigma, I would like to do the following: Have three regions in the physical address space (not necessarily memory): i) one for bootstrap, fiasco ii) moe, ned, sigma, l4linux iii). Special region accessible only from (i) - bootstrap and fiasco. If it is not possible to confine sigma, i.e., hide a portion of the address space from it, then add sigma to lists (i,iii) and remove it from (ii) - and so on for the other modules. If it is moe, could my goal be reached by adding an IO device as a blocker but then not give moe the corresponding capability? In the stack moe is below any notion of an IO device, so that would not work. Adam -- Adam a...@os.inf.tu-dresden.de Lackorzynski http://os.inf.tu-dresden.de/~adam/ ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
RE: Aborted L4linux due to CLI/STI
hi, a. Ok I will re-enable the cmpxchg8b. b. I am not sure about the changes and since it occurs quite rarely, I am not sure if it is a case of not cleaning up interrupts properly after a crash. I will live with them for now and get back to this once the other issues are sorted out. thanks, Ramya From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Adam Lackorzynski [a...@os.inf.tu-dresden.de] Sent: 09 September 2014 22:46 To: l4-hackers@os.inf.tu-dresden.de Subject: Re: Aborted L4linux due to CLI/STI Hi, On Mon Sep 08, 2014 at 07:10:00 +, Masti Ramya Jayaram wrote: That's ok as long it's also ok for native Linux. I am not sure what they did with native Linux but I know from the SCC forums that these instructions do not exist. So I guess that is ok. I'm wondering how did you end up with the code containing the cli? In L4Linux this part is replaced. Well, I am not sure about that. I did force a couple of options though as I already mentioned. The reason was that when I tried to compile L4linux for pentium/i586, I saw that it would complain about CONFIG_X86_CMPXCHG64 not being enabled (this corresponds to the PAE option). So I disabled this explicitly since the processors on the SCC do not contain PAE. I also disabled the CONFIG_X86_CMPXCHG and CONFIG_CMPXCHG_LOCAL options. As far as I know the SCC does have a cmpxchg8b so you could just make Linux use that by tweaking the config. Attached is my l4linux config. Just as a note, I am using quite an old build of L4linux. Do you know of any related bugs? Uh, that's old. There are probably many. Were there any significant changes to that version? Adam -- Adam a...@os.inf.tu-dresden.de Lackorzynski http://os.inf.tu-dresden.de/~adam/ ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
RE: Physical memory allocation to L4linux
Update: Sorry, I misunderstood a few things in the previous email. The change
was in fiasco and not sigma and therefore, in theory, no user space process
should have access to the protected region. This also implies that sigma0 will
not have access if I understand correctly which is rather nice.
So, I tried changing the handle_sigma0_page_fault function. This is located in
kernel/fiasco/src/kern/ia32/thread-ia32.cpp to prevent mapping of some range of
physical memory (NOT RAM) as follows:
IMPLEMENT inline
bool
Thread::handle_sigma0_page_fault(Address pfa)
{
size_t size;
// Check if mapping a superpage doesn't exceed the size of physical memory
if (Cpu::have_superpages()
// Some distributions do not allow to mmap below a certain threshold
// (like 64k on Ubuntu 8.04) so we cannot map a superpage at 0 if
// we're Fiasco-UX
(!Config::Is_ux || !(pfa Config::SUPERPAGE_SIZE)))
{
pfa = Config::SUPERPAGE_MASK;
size = Config::SUPERPAGE_SIZE;
}
else
{
pfa = Config::PAGE_MASK;
size = Config::PAGE_SIZE;
}
--
- My change start
if((pfa=0xe000) (pfa=0xefff)){
printf(\nNot allowed...denying\n);
return Mem_space::Insert_err_nomem;
}
--
- My change end
return mem_space()-v_insert(Mem_space::Phys_addr(pfa), Mem_space::Addr(pfa),
Mem_space::Size(size),
Mem_space::Page_writable
| Mem_space::Page_user_accessible)
!= Mem_space::Insert_err_nomem;
}
However, I realized that when I try to map this address space using
l4io_request_iomem, I see that it goes through and the call never reaches until
fiasco's sigma0_page_fault handler. A little more debugging revealed that it
goes as far as the following function in l4/pkg/sigma0/server/src/memmap.cc:
static
void map_mem(l4_fpage_t fp, Memory_type fn, l4_umword_t t, Answer *an)
{
an-clear();
Mem_man *m;
switch (fn)
{
case Ram:
m = Mem_man::ram();
break;
case Io_mem:
case Io_mem_cached:
m = iomem;
break;
default:
return;
}
unsigned long addr = m-alloc(Region::bs(fp.raw ~((1UL 12) - 1),
1UL l4_fpage_size(fp), t));
if (addr == ~0UL)
return;
/* the Fiasco kernel makes the page non-cachable if the frame
* address is greater than mem_high */
an-snd_base(addr);
an-snd_fpage(addr, l4_fpage_size(fp), false, fn != Io_mem);
an-tag = l4_msgtag(0, 0, 1, 0);
return;
}
From here on, the generics make it a little hard to parse. So I am not sure
where the control flow ends but it does not get to fiasco's sigma0 page fault
handler. Given this,
a. does the handle_sigma0_page_fault only work on memory. The region I want to
protect is not RAM.
b. Given that the setup of sigma0's memory happens in
kernel/fiasco/src/kern/kernel_thread-std.cpp, is there more to be done here?
Thanks.
Ramya
From: Masti Ramya Jayaram
Sent: 10 September 2014 09:46
To: Adam Lackorzynski; l4-hackers@os.inf.tu-dresden.de
Subject: RE: Physical memory allocation to L4linux
Hey Adam,
thanks a lot for the replies. I still have a few questions/clarifications
though. :-)
a. Is there a way to implement the MODE=sigma for linux? I could not find
anything that points to it.
b. Now if I were to tweak the page fault handler in sigma, then does that imply
that even a corrupt moe/ned/l4linux cannot get access to the protected part of
the address space?
c. Who are the clients of this page fault handler? Is it just what runs on top
of sigma like moe, ned, l4linux or even faisco?
d. If it is just moe, ned, l4linux, then can I use the same page fault trick to
ensure that (moe,ned,sigma) they always get memory from a certain dedicated
region?
Thanks a ton for your patience (to everyone on the list and particularly you
and Martin). I totally appreciate it.
Best,
Ramya
From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Adam
Lackorzynski [a...@os.inf.tu-dresden.de]
Sent: 09 September 2014 23:16
To: l4-hackers@os.inf.tu-dresden.de
Subject: Re: Physical memory allocation to L4linux
On Tue Sep 09, 2014 at 16:03:11 +, Masti Ramya Jayaram wrote:
I would like to explain my situation better. Here is what I intend:
a. What is the lowest module (bootstrap, fiasco, sigma, moe,
ned,l4linux) that can be confined not access a portion of the address
space?
Without changing anything and with this list of program, it's just
L4Linux. Reason is that ned typically has a cap to sigma0 because it
needs to give it to io.
When you would like to make a little change, in Fiasco
Physical memory allocation to L4linux
Hi all, I start l4linux by specifying 64M on memory in its configuration file. I would like this memory to be a fixed range of physical memory and I was wondering if there is a way to do it. I understand from my earlier posts that the image of l4linux is in the same location everytime because the boot loader puts it there. I am not referring to this memory. I am concerned with the 64M that will potentially get allocated to l4linux. Thanks in advance, ramya ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
RE: Physical memory allocation to L4linux
Hey Martin,
thanks for that tip. How do I use this device for L4linux though (and only for
it)? I am not sure where memory allocation occurs when L4linux faults or if it
is pre-allocated completely to L4linux.
Best,
ramya
From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Martin
Schröder [martin.schroe...@openlimit.com]
Sent: 09 September 2014 11:56
To: l4-hackers@os.inf.tu-dresden.de
Subject: Re: Physical memory allocation to L4linux
Hi Ramya,
Am 09.09.2014 um 11:39 schrieb Masti Ramya Jayaram:
Hi all,
I start l4linux by specifying 64M on memory in its configuration file. I
would like this memory to be a fixed range of physical memory and I was
wondering if there is a way to do it.
MYRAM = new Device() {
.hid= myram;
new-res Mmio(0x1000 .. 0x13FF, 0xC000);
}
[http://os.inf.tu-dresden.de/pipermail/l4-hackers/2014/006617.html]
Best regards,
Martin.
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
Preventing access to a part of the address space from Sigma0
Dear all, I would like to prevent access to a part of the address space from sigma0 for security reasons on my board. For example, I would like to prevent even sigma from accessing say 0xc000 - 0x of memory. What is the best way to accomplish this and test it? Thanks, ramya ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
RE: Checking executables before running them in L4re/fiasco
Hey Adam,
Yes, I explicitly made sure that there are no extra characters by setting the
entire page of 4096 bytes to zero before copying over the new config. I am not
sure what you mean by changed the target - I simply replace the config file. So
I thought it picks up the size information from the MBI header.
I will try the MODE option as you mentioned it. I misunderstood it earlier.
Thanks,
Ramya
From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Adam
Lackorzynski [a...@os.inf.tu-dresden.de]
Sent: 08 September 2014 00:03
To: l4-hackers@os.inf.tu-dresden.de
Subject: Re: Checking executables before running them in L4re/fiasco
Hi,
On Fri Sep 05, 2014 at 12:29:15 +, Masti Ramya Jayaram wrote:
Thanks for the information. I have two issues/questions:
a. I looked at the move_modules (which does not do much in my case) and I
tried to replace the linux.cfg from
Compiled in version:
--
--vim:set ft=lua:
L4.default_loader:start({ caps = {
log = L4.Env.log:m(rws),
},
l4re_dbg = L4.Dbg.Warn,
log = { l4linux, yellow },
}, rom/vmlinuz mem=128M console=ttyLv0,115200 l4x_rd=rom/ramdisk-x86.rd
root=1:0 ramdisk_size=4000 l4ser.vkey_enable=1);
Replaced version: (just the memory changes but this results in a difference
in the length!)
---
--vim:set ft=lua:
L4.default_loader:start({ caps = {
log = L4.Env.log:m(rws),
},
l4re_dbg = L4.Dbg.Warn,
log = { l4linux, yellow },
}, rom/vmlinuz mem=64M console=ttyLv0,115200 l4x_rd=rom/ramdisk-x86.rd
root=1:0 ramdisk_size=4000 l4ser.vkey_enable=1);
I achieve this with the following code that runs immediately after
construct_mbi:
a. Copy the placed version into the start address of the linux_cfg
module (this happens correctly as I can see the memory dump).
b. Change the compressed and uncompressed lengths in mod_info to
reflect the smaller size.
The process runs until ned which fails to parse the config file
(unexpected symbol error) and the problem seems to be the difference
in length of the two configs. For example, if the 2nd config is
mem=064M instead of mem=64M, then it works. I change the length
appropriately in mod_info of the mbi. What am I missing?
When you already changed the target, changing the size variables
probably won't make a difference. Did you clear out the old contents? I
think you have remaining characters in the file that are interpreted and
causing the error.
a. The MODE = sigma0 suggestion: I tried this by changing
bootstrap/server/src/Make.rules but I get an error saying: ld: cannot
find -ll4util. The relevant part in the linker line that differs from
the default working build (with MODE = lib) is:
Wait. I meant the MODE=sigma0 for newly created programs. If you want to
make bootstrap smaller, you need to remove code/features from it.
Adam
--
Adam a...@os.inf.tu-dresden.de
Lackorzynski http://os.inf.tu-dresden.de/~adam/
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
RE: Aborted L4linux due to CLI/STI
Hey Adam, That's ok as long it's also ok for native Linux. I am not sure what they did with native Linux but I know from the SCC forums that these instructions do not exist. So I guess that is ok. I'm wondering how did you end up with the code containing the cli? In L4Linux this part is replaced. Well, I am not sure about that. I did force a couple of options though as I already mentioned. The reason was that when I tried to compile L4linux for pentium/i586, I saw that it would complain about CONFIG_X86_CMPXCHG64 not being enabled (this corresponds to the PAE option). So I disabled this explicitly since the processors on the SCC do not contain PAE. I also disabled the CONFIG_X86_CMPXCHG and CONFIG_CMPXCHG_LOCAL options. Attached is my l4linux config. Just as a note, I am using quite an old build of L4linux. Do you know of any related bugs? Thanks, Ramya From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Adam Lackorzynski [a...@os.inf.tu-dresden.de] Sent: 08 September 2014 00:29 To: l4-hackers@os.inf.tu-dresden.de Subject: Re: Aborted L4linux due to CLI/STI On Fri Sep 05, 2014 at 16:21:34 +, Masti Ramya Jayaram wrote: I forgot to mention that it also does not support the traditional cmpxchg instructions using the lock line. That is why I disabled it. That's ok as long it's also ok for native Linux. On 05 Sep 2014, at 18:16, Masti Ramya Jayaram rma...@inf.ethz.ch wrote: Dear all, I get an error during L4linux boot when I enter the debugger (using ESC) and resume: Aborted L4linux due to CLI/STI at 0x534911. When I disassemble this using gdb: (gdb) disassemble 0x534911 Dump of assembler code for function cmpxchg8b_emu: 0x00534910 +0:pushf 0x00534911 +1:cli 0x00534912 +2:cmp(%esi),%eax 0x00534914 +4:jne0x534922 cmpxchg8b_emu+18 0x00534916 +6:cmp0x4(%esi),%edx 0x00534919 +9:jne0x534924 half_same 0x0053491b +11:mov%ebx,(%esi) 0x0053491d +13:mov%ecx,0x4(%esi) 0x00534920 +16:popf 0x00534921 +17:ret 0x00534922 +18:mov(%esi),%eax 0x00534924 +0:mov0x4(%esi),%edx 0x00534927 +3:popf 0x00534928 +4:ret When I built l4linux, I had to disable CONFIG_X86_CMPXCHG, CONFIG_CMPXCHG_LOCAL and CONFIG_X86_CMPXCHG64 despite building for the Pentium because my processor does not support PAE (which is the option that enables this). I am guessing This is the reason it tries to emulate the CMPXCHG instructions. What is the correct l4linux configuration to use for a machine that does not support any of the atomic instructions like CMPXCHG8, ATOMIC, etc.? Which processor type did you select? In L4Linux, there's an adapted version of the cmpxchg8b_emu code that is enabled if CONFIG_X86_CMPXCHG64 is not set. I'm wondering how did you end up with the code containing the cli? In L4Linux this part is replaced. Adam -- Adam a...@os.inf.tu-dresden.de Lackorzynski http://os.inf.tu-dresden.de/~adam/ ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers# # Automatically generated make config: don't edit # Linux/l4 2.6.38-l4 Kernel Configuration # Fri Sep 5 02:47:55 2014 # # # L4Linux configuration # CONFIG_L4=y CONFIG_L4_LINUX=y CONFIG_L4_OBJ_TREE=/home/rmasti/fiasco/r33-new/src/l4/build CONFIG_L4_ARCH_X86=y # CONFIG_L4_ARCH_ARM is not set CONFIG_L4_VMLINUZ_SUFFIX= CONFIG_HZ=100 CONFIG_PHYSICAL_START=0x0 # # Features # # CONFIG_L4_SERVER is not set # # IRQ Handling # # CONFIG_L4_IRQ_SINGLE is not set CONFIG_L4_IRQ_MULTI=y # # Stub drivers # CONFIG_L4_EXTERNAL_RTC=y CONFIG_L4_BLK_DS_DRV=y CONFIG_L4_CHR_DS_DRV=y # CONFIG_L4_FB_DRIVER is not set # CONFIG_L4_NET_ANKH_DRV is not set # CONFIG_L4_NET_L4SHM is not set CONFIG_L4_SERIAL=y CONFIG_L4_SERIAL_CONSOLE=y # CONFIG_L4_SERIAL_SHM is not set # # Debugging options # # CONFIG_L4_DEBUG is not set # # Advanced options # CONFIG_L4_MEMSIZE=32 CONFIG_L4_TAMED=y # CONFIG_L4_VCPU is not set # # Priority configuration # CONFIG_L4_PRIO_SERVER_PROC=1 CONFIG_L4_PRIO_IDLER=2 CONFIG_L4_PRIO_IRQ_BASE=3 CONFIG_L4_PRIO_TAMER=22 CONFIG_L4_PRIO_L4FB_INPUT=7 CONFIG_L4_PRIO_L4FB_REFRESH=7 CONFIG_L4_PRIO_L4ANKH=7 CONFIG_L4_CONFIG_CHECKS=y # CONFIG_L4_SAMPLE_MODULE is not set # # # # CONFIG_64BIT is not set CONFIG_X86_32=y # CONFIG_X86_64 is not set CONFIG_X86=y CONFIG_INSTRUCTION_DECODER=y CONFIG_OUTPUT_FORMAT=elf32-i386 CONFIG_ARCH_DEFCONFIG=arch/x86/configs/i386_defconfig CONFIG_GENERIC_CMOS_UPDATE=y CONFIG_CLOCKSOURCE_WATCHDOG=y CONFIG_GENERIC_CLOCKEVENTS=y CONFIG_LOCKDEP_SUPPORT=y CONFIG_STACKTRACE_SUPPORT=y CONFIG_HAVE_LATENCYTOP_SUPPORT=y CONFIG_MMU=y CONFIG_ZONE_DMA=y # CONFIG_NEED_DMA_MAP_STATE is not set CONFIG_NEED_SG_DMA_LENGTH=y CONFIG_GENERIC_ISA_DMA=y CONFIG_GENERIC_IOMAP=y
RE: Adding a custom library to l4/pkg/bootstrap
Hey Martin,
No I did not have the require_libs declared. I will try that.
Thanks,
ramya
From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Martin
Küttler [martin.kuett...@gmail.com]
Sent: 04 September 2014 09:27
To: l4-hackers@os.inf.tu-dresden.de
Subject: Re: Adding a custom library to l4/pkg/bootstrap
On 09/03/2014 09:51 AM, Masti Ramya Jayaram wrote:
Hey Jan,
Sorry for underspecifying it. Here are the details.
I have a library called scc-sec under the l4/pkg directory. It has one file:
a. init.c that contains
void hello(){
printf(hello);
}
I compile this as a library and I can see the scc-sec.so and scc-sec.a in the
l4/build/pkg/scc-sec directory
Now in l4/pkg/bootstrap:
a. Control file
requires: (added) scc-sec
b. Make.rules:
-lscc-sec
Is -lscc-sec the literal content of this file? That doesn't do anything.
c. startup.cc
I declare the function as:
extern C {
void hello();
}
and somewhere in startup() function, I call it
hello();
On trying to compile this, I get (exact error as it is compiled)
in path_to-startup.cc: undefined reference to function hello()
I do not think it is the printf because it explicitly gives me an undefined
reference to hello :)
To me this looks like you don't actually link to your scc-sec library.
Does your Makefile for bootstrap include something like
REQUIRES_LIBS = scc_sec
?
Martin
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
RE: Checking executables before running them in L4re/fiasco
Hey Adam,
Thanks for the information. I have two issues/questions:
a. I looked at the move_modules (which does not do much in my case) and I tried
to replace the linux.cfg from
Compiled in version:
--
--vim:set ft=lua:
L4.default_loader:start({ caps = {
log = L4.Env.log:m(rws),
},
l4re_dbg = L4.Dbg.Warn,
log = { l4linux, yellow },
}, rom/vmlinuz mem=128M console=ttyLv0,115200 l4x_rd=rom/ramdisk-x86.rd
root=1:0 ramdisk_size=4000 l4ser.vkey_enable=1);
Replaced version: (just the memory changes but this results in a difference in
the length!)
---
--vim:set ft=lua:
L4.default_loader:start({ caps = {
log = L4.Env.log:m(rws),
},
l4re_dbg = L4.Dbg.Warn,
log = { l4linux, yellow },
}, rom/vmlinuz mem=64M console=ttyLv0,115200 l4x_rd=rom/ramdisk-x86.rd
root=1:0 ramdisk_size=4000 l4ser.vkey_enable=1);
I achieve this with the following code that runs immediately after
construct_mbi:
a. Copy the placed version into the start address of the linux_cfg module (this
happens correctly as I can see the memory dump).
b. Change the compressed and uncompressed lengths in mod_info to reflect the
smaller size.
The process runs until ned which fails to parse the config file (unexpected
symbol error) and the problem seems to be the difference in length of the two
configs. For example, if the 2nd config is mem=064M instead of mem=64M,
then it works. I change the length appropriately in mod_info of the mbi. What
am I missing?
a. The MODE = sigma0 suggestion: I tried this by changing
bootstrap/server/src/Make.rules but I get an error saying: ld: cannot find
-ll4util. The relevant part in the linker line that differs from the default
working build (with MODE = lib) is:
== Linking bootstrap.elf
..
-Tbootstrap.ld --start-group -ll4util -ll4sys -luc_c -static -nostdlib
-ldrivers_uart -lcxx_base -lcxx_io -luc_c
/usr/lib/gcc/x86_64-linux-gnu/4.4.3/32/libgcc.a --end-group --warn-common
-Ttext=0x2d /usr/lib/gcc/x86_64-linux-gnu/4.4.3/32//crtend.o
/home/rmasti/fiasco/r33-new/src/l4/build/lib/x86_pentium/crtn.o
The linker line with MODE = lib (the difference seems to be that l4util,
l4sys, etc are not really there in the commandline)
== Linking bootstrap.elf
..
-Tbootstrap.ld --start-group-static -nostdlib -ldrivers_uart -lcxx_base
-lcxx_io -luc_c /usr/lib/gcc/x86_64-linux-gnu/4.4.3/32/libgcc.a --end-group
--warn-common -Ttext=0x2d /usr/lib/gcc/x86_64-linux-gnu/4.4.3/32//crtend.o
/home/rmasti/fiasco/r33-new/src/l4/build/lib/x86_pentium/crtn.o
Why does this happen? What is the relevant modules.list entry for this mode?
Thanks,
ramya
From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Adam
Lackorzynski [a...@os.inf.tu-dresden.de]
Sent: 05 September 2014 00:39
To: l4-hackers@os.inf.tu-dresden.de
Subject: Re: Checking executables before running them in L4re/fiasco
││ │
│ ││ │
│ ││ │
│ ││ │
│ ││ │
│ ││ │
│ ││
On Wed Sep 03, 2014 at 14:43:59 +, Masti Ramya Jayaram wrote:
You mean binaries that are started?
Yes but some modules have dependencies (like the Io server and
examples of target dependencies include libsigma). Is there a way to
find all of them?
Ok, you mean all the libraries that are linked to a program. One way to
find that out is to build the program with make V=1, see the linker
step and look for all the -lxxx options. The lib directory in the build
tree has all the libraries and those are links to the actual package
directory so you see the package there.
Finally, on a related note, could you explain to me how all the
modules get merged to create the elfimage? I ask because I have to
access the modules (vmlinux, l4linux.cfg and ramdisk) from bootstrap
in order to modify/check them. For example, I would like to load the
modules encrypted and decrypt them in bootstrap before continuing. I
can do decryption in place but again, I am not sure about module
sizes, etc.
I see that some of this information is included during the image
creation process in l4/pkg/bootstrap/server/src/build.pl but I cannot
quite get the entire picture. Could you point me to any documentation
about it?
In bootstrap, there's a move_modules() function that iterates over all
the modules and should show you how how to do that.
Adam
--
Adam a...@os.inf.tu-dresden.de
Lackorzynski http://os.inf.tu-dresden.de/~adam
Aborted L4linux due to CLI/STI
Dear all, I get an error during L4linux boot when I enter the debugger (using ESC) and resume: Aborted L4linux due to CLI/STI at 0x534911. When I disassemble this using gdb: (gdb) disassemble 0x534911 Dump of assembler code for function cmpxchg8b_emu: 0x00534910 +0: pushf 0x00534911 +1: cli 0x00534912 +2: cmp(%esi),%eax 0x00534914 +4: jne0x534922 cmpxchg8b_emu+18 0x00534916 +6: cmp0x4(%esi),%edx 0x00534919 +9: jne0x534924 half_same 0x0053491b +11:mov%ebx,(%esi) 0x0053491d +13:mov%ecx,0x4(%esi) 0x00534920 +16:popf 0x00534921 +17:ret 0x00534922 +18:mov(%esi),%eax 0x00534924 +0: mov0x4(%esi),%edx 0x00534927 +3: popf 0x00534928 +4: ret When I built l4linux, I had to disable CONFIG_X86_CMPXCHG, CONFIG_CMPXCHG_LOCAL and CONFIG_X86_CMPXCHG64 despite building for the Pentium because my processor does not support PAE (which is the option that enables this). I am guessing This is the reason it tries to emulate the CMPXCHG instructions. What is the correct l4linux configuration to use for a machine that does not support any of the atomic instructions like CMPXCHG8, ATOMIC, etc.? Thanks, ramya ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
Re: Aborted L4linux due to CLI/STI
I forgot to mention that it also does not support the traditional cmpxchg instructions using the lock line. That is why I disabled it. Sorry about the multiple emails, Ramya Sent from my phone On 05 Sep 2014, at 18:16, Masti Ramya Jayaram rma...@inf.ethz.ch wrote: Dear all, I get an error during L4linux boot when I enter the debugger (using ESC) and resume: Aborted L4linux due to CLI/STI at 0x534911. When I disassemble this using gdb: (gdb) disassemble 0x534911 Dump of assembler code for function cmpxchg8b_emu: 0x00534910 +0:pushf 0x00534911 +1:cli 0x00534912 +2:cmp(%esi),%eax 0x00534914 +4:jne0x534922 cmpxchg8b_emu+18 0x00534916 +6:cmp0x4(%esi),%edx 0x00534919 +9:jne0x534924 half_same 0x0053491b +11:mov%ebx,(%esi) 0x0053491d +13:mov%ecx,0x4(%esi) 0x00534920 +16:popf 0x00534921 +17:ret 0x00534922 +18:mov(%esi),%eax 0x00534924 +0:mov0x4(%esi),%edx 0x00534927 +3:popf 0x00534928 +4:ret When I built l4linux, I had to disable CONFIG_X86_CMPXCHG, CONFIG_CMPXCHG_LOCAL and CONFIG_X86_CMPXCHG64 despite building for the Pentium because my processor does not support PAE (which is the option that enables this). I am guessing This is the reason it tries to emulate the CMPXCHG instructions. What is the correct l4linux configuration to use for a machine that does not support any of the atomic instructions like CMPXCHG8, ATOMIC, etc.? Thanks, ramya ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
RE: Adding a custom library to l4/pkg/bootstrap
Hey Jan,
Sorry for underspecifying it. Here are the details.
I have a library called scc-sec under the l4/pkg directory. It has one file:
a. init.c that contains
void hello(){
printf(hello);
}
I compile this as a library and I can see the scc-sec.so and scc-sec.a in the
l4/build/pkg/scc-sec directory
Now in l4/pkg/bootstrap:
a. Control file
requires: (added) scc-sec
b. Make.rules:
-lscc-sec
c. startup.cc
I declare the function as:
extern C {
void hello();
}
and somewhere in startup() function, I call it
hello();
On trying to compile this, I get (exact error as it is compiled)
in path_to-startup.cc: undefined reference to function hello()
I do not think it is the printf because it explicitly gives me an undefined
reference to hello :)
Thanks in advance for helping with this,
Ramya
From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Jan
Bierbaum [jan.bierb...@os.inf.tu-dresden.de]
Sent: 02 September 2014 21:56
To: l4-hackers@os.inf.tu-dresden.de
Subject: Re: Adding a custom library to l4/pkg/bootstrap
On 02.09.2014 13:32, Masti Ramya Jayaram wrote:
However, now I am trying to call a function in the library through an
included header file (which is part of the library) and it fails
(undefined reference).
Could you give the *exact* error message? BTW, this is something you
should always do if you want people to be able to help you ;-)
My guess at this point would be that the undefined reference is not your
'hello' function but 'printf'...
I made sure that I am indeed using the extern C directive. I also
tried putting the function declaration in here instead of the header
file itself but it does not help.
The function declaration is not the problem here. If it was, the
compiler would give you an error along the lines of 'hello' was not
declared in this scope.
Regards, Jan
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
RE: Checking executables before running them in L4re/fiasco
For a start, you can set MODE = sigma0 in the Makefile to get a much
Ok, I will try the MODE=sigma0.
You mean binaries that are started?
Yes but some modules have dependencies (like the Io server and examples of
target dependencies include libsigma). Is there a way to find all of them?
Finally, on a related note, could you explain to me how all the modules get
merged to create the elfimage? I ask because I have to access the modules
(vmlinux, l4linux.cfg and ramdisk) from bootstrap in order to modify/check
them. For example, I would like to load the modules encrypted and decrypt them
in bootstrap before continuing. I can do decryption in place but again, I am
not sure about module sizes, etc.
I see that some of this information is included during the image creation
process in l4/pkg/bootstrap/server/src/build.pl but I cannot quite get the
entire picture. Could you point me to any documentation about it?
Thanks,
Ramya
From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Adam
Lackorzynski [a...@os.inf.tu-dresden.de]
Sent: 03 September 2014 00:02
To: l4-hackers@os.inf.tu-dresden.de
Subject: Re: Checking executables before running them in L4re/fiasco
On Tue Sep 02, 2014 at 14:08:03 +, Masti Ramya Jayaram wrote:
Also, could you elaborate a bit on how one could have smaller
components? I decided to do it at bootstrap at the moment because I
realized that using the IO server also means including the entire C
library (and much more).
For a start, you can set MODE = sigma0 in the Makefile to get a much
simpler setup (don't be confused by the name). Of course, not all
features are there then but should be ok for checking a binary. More
libraries can be removed further but also requires to do more by hand
then.
On a related note, is there a way to find all the packages in use
(there is more than what is compiled by module.list I can see)?
You mean binaries that are started? modules.list has defaults for
kernel, sigma0 and roottask but otherwise everything else needs to be
put there.
From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Masti
Ramya Jayaram [rma...@inf.ethz.ch]
Sent: 02 September 2014 09:57
To: Adam Lackorzynski; l4-hackers@os.inf.tu-dresden.de
Subject: RE: Checking executables before running them in L4re/fiasco
Thanks for the suggestions. From what you say, I guess it is best to
do something between ned and say vmlinuz. It also makes better sense
semantically (the kernel has booted and checks if the application is
ok).
Is there a way to accomplish the following:
a. Ned starts up a new process which does some checks and returns OK/STOP.
b. depending on the value returned by the new process, ned decides to
launch/stop vmlinuz.
When I try roottask moe rom/hello rom/vmlinuz in the modules.list
file, they execute in parallel. Is there a way to make it sequential?
You start ned via moe (as usual) and have a script like this:
local l = L4.default_loader;
local e = l:start({}, rom/checker rom/hello);
if e:wait() == 0 then
l:start({}, rom/hello);
else
print(Something is wrong with hello.);
end
So the checker program does exit(0) or exit(1) depending on its result.
Adam
--
Adam a...@os.inf.tu-dresden.de
Lackorzynski http://os.inf.tu-dresden.de/~adam/
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
RE: Checking executables before running them in L4re/fiasco
Thanks for the suggestions. From what you say, I guess it is best to do something between ned and say vmlinuz. It also makes better sense semantically (the kernel has booted and checks if the application is ok). Is there a way to accomplish the following: a. Ned starts up a new process which does some checks and returns OK/STOP. b. depending on the value returned by the new process, ned decides to launch/stop vmlinuz. When I try roottask moe rom/hello rom/vmlinuz in the modules.list file, they execute in parallel. Is there a way to make it sequential? Thanks, Ramya From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Adam Lackorzynski [a...@os.inf.tu-dresden.de] Sent: 01 September 2014 23:57 To: l4-hackers@os.inf.tu-dresden.de Subject: Re: Checking executables before running them in L4re/fiasco Hi, On Mon Sep 01, 2014 at 07:54:02 +, Masti Ramya Jayaram wrote: Thanks for the suggestion. I have one other constraint: I would like to keep trusted computing base (or the amount of security critical code as small as possible). From what I understand from http://l4re.org/doc/index.html, the following constitutes the minimal trusted code (which if buggy or compromised by an attacker ruins the isolation properties of the kernel). a. Whole of fiasco b. Sigma0 (the root pager) c. Moe (the loader) d. Ned (the first loaded program used to bootstrap the rest) In this setup, yes. Potentially one could have specialized components that are even smaller. The IO server is not really security critical as far as I understood. Is this correct? IO has access to the hardware, so it can do interesting things. So I would ideally like to do it even before sigma because if the checks fail, I do not want to proceed and it would be ok to do it in moe or ned. So could you elaborate on mmap part to map a piece of IO memory say in a separate executable after sigma? If you want to do it so early I suggest doing it even earlier in bootstrap. There you run on physical memory and can directly access anything. You're bit limited feature-wise there but for checking some memory regions (aka the binaries) it should be ok. Stuffing another binary inbetween sigma0 and moe is not directly easy and is also quite limited feature-wise. Adam -- Adam a...@os.inf.tu-dresden.de Lackorzynski http://os.inf.tu-dresden.de/~adam/ ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
Adding a custom library to l4/pkg/bootstrap
Hi all, I have managed to compile a custom C library (libscc-sec) as part of the l4 packages. Now I would like to use it in the bootstrap package (make a call to a function in the library). I did the following: a. Added the library as a requirement in pkg/bootstrap/Control requires: drivers_uart drivers_of libc l4util cxx_io libscc-sec b. Modified the pkg/bootstrap/server/src/Make.rules so that it also looks for libscc-sec by: L4_LIBS = -static -nostdlib $(DRV_LIBS) -lcxx_base -lcxx_io -llibscc-sec But on compiling, I get an error that says that -llibscc-sec not found The package is getting compiled and I can see the library libscc-sec.a in the build folder. What am I missing? Thanks, ramya ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
RE: Fault in function writev (ulibc)
No, it would occur randomly. I realized I had some mismatch in function prototypes with arrays. Fixing that got rid of the problem. Thanks, Ramya From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Adam Lackorzynski [a...@os.inf.tu-dresden.de] Sent: 02 September 2014 00:00 To: l4-hackers@os.inf.tu-dresden.de Subject: Re: Fault in function writev (ulibc) On Mon Sep 01, 2014 at 16:37:44 +, Masti Ramya Jayaram wrote: I am running a custom application package on fiasco using a serial line (no keyboard/screen) and I intermittently get faults caused by printf. I notice that on disassembling the code around the error that the function is called writev in ulibc. Has anyone experienced something similar before? Not that I could remember. Just happens for a standard printf(foo\n)? Adam -- Adam a...@os.inf.tu-dresden.de Lackorzynski http://os.inf.tu-dresden.de/~adam/ ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
RE: Adding a custom library to l4/pkg/bootstrap
Sorry about the multiple emails. I managed to get past the library not found
error - it was a matter of nomenclature.
However, now I am trying to call a function in the library through an included
header file (which is part of the library) and it fails (undefined reference).
My header file: init.h
#define BLAH 10
int abcd = 5;
int hello();
Code in my init.c
int hello(){
printf(Hello);
}
My C++ code:
extern C{
#include l4/libscc-sec/init.h
}
int main(){
printf(Vars from lib %x, %x, BLAH. abcd)
hello();
}
The program fails to compile with the calls to the function hello. If I remove
hello, it compiles.
I made sure that I am indeed using the extern C directive. I also tried
putting the function declaration in here instead of the header file itself but
it does not help.
Thanks,
ramya
From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Masti
Ramya Jayaram [rma...@inf.ethz.ch]
Sent: 02 September 2014 11:37
To: l4-hackers@os.inf.tu-dresden.de
Subject: Adding a custom library to l4/pkg/bootstrap
Hi all,
I have managed to compile a custom C library (libscc-sec) as part of the l4
packages. Now I would like to use it in the bootstrap package (make a call to a
function in the library). I did the following:
a. Added the library as a requirement in pkg/bootstrap/Control
requires: drivers_uart drivers_of libc l4util cxx_io libscc-sec
b. Modified the pkg/bootstrap/server/src/Make.rules so that it also looks for
libscc-sec by:
L4_LIBS = -static -nostdlib $(DRV_LIBS) -lcxx_base -lcxx_io
-llibscc-sec
But on compiling, I get an error that says that -llibscc-sec not found
The package is getting compiled and I can see the library libscc-sec.a in the
build folder. What am I missing?
Thanks,
ramya
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
RE: Checking executables before running them in L4re/fiasco
Also, could you elaborate a bit on how one could have smaller components? I decided to do it at bootstrap at the moment because I realized that using the IO server also means including the entire C library (and much more). On a related note, is there a way to find all the packages in use (there is more than what is compiled by module.list I can see)? Thanks, ramya From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Masti Ramya Jayaram [rma...@inf.ethz.ch] Sent: 02 September 2014 09:57 To: Adam Lackorzynski; l4-hackers@os.inf.tu-dresden.de Subject: RE: Checking executables before running them in L4re/fiasco Thanks for the suggestions. From what you say, I guess it is best to do something between ned and say vmlinuz. It also makes better sense semantically (the kernel has booted and checks if the application is ok). Is there a way to accomplish the following: a. Ned starts up a new process which does some checks and returns OK/STOP. b. depending on the value returned by the new process, ned decides to launch/stop vmlinuz. When I try roottask moe rom/hello rom/vmlinuz in the modules.list file, they execute in parallel. Is there a way to make it sequential? Thanks, Ramya From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Adam Lackorzynski [a...@os.inf.tu-dresden.de] Sent: 01 September 2014 23:57 To: l4-hackers@os.inf.tu-dresden.de Subject: Re: Checking executables before running them in L4re/fiasco Hi, On Mon Sep 01, 2014 at 07:54:02 +, Masti Ramya Jayaram wrote: Thanks for the suggestion. I have one other constraint: I would like to keep trusted computing base (or the amount of security critical code as small as possible). From what I understand from http://l4re.org/doc/index.html, the following constitutes the minimal trusted code (which if buggy or compromised by an attacker ruins the isolation properties of the kernel). a. Whole of fiasco b. Sigma0 (the root pager) c. Moe (the loader) d. Ned (the first loaded program used to bootstrap the rest) In this setup, yes. Potentially one could have specialized components that are even smaller. The IO server is not really security critical as far as I understood. Is this correct? IO has access to the hardware, so it can do interesting things. So I would ideally like to do it even before sigma because if the checks fail, I do not want to proceed and it would be ok to do it in moe or ned. So could you elaborate on mmap part to map a piece of IO memory say in a separate executable after sigma? If you want to do it so early I suggest doing it even earlier in bootstrap. There you run on physical memory and can directly access anything. You're bit limited feature-wise there but for checking some memory regions (aka the binaries) it should be ok. Stuffing another binary inbetween sigma0 and moe is not directly easy and is also quite limited feature-wise. Adam -- Adam a...@os.inf.tu-dresden.de Lackorzynski http://os.inf.tu-dresden.de/~adam/ ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
RE: Checking executables before running them in L4re/fiasco
Hey Adam, Thanks for the suggestion. I have one other constraint: I would like to keep trusted computing base (or the amount of security critical code as small as possible). From what I understand from http://l4re.org/doc/index.html, the following constitutes the minimal trusted code (which if buggy or compromised by an attacker ruins the isolation properties of the kernel). a. Whole of fiasco b. Sigma0 (the root pager) c. Moe (the loader) d. Ned (the first loaded program used to bootstrap the rest) The IO server is not really security critical as far as I understood. Is this correct? So I would ideally like to do it even before sigma because if the checks fail, I do not want to proceed and it would be ok to do it in moe or ned. So could you elaborate on mmap part to map a piece of IO memory say in a separate executable after sigma? Thanks in advance, Ramya From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Adam Lackorzynski [a...@os.inf.tu-dresden.de] Sent: 01 September 2014 00:14 To: l4-hackers@os.inf.tu-dresden.de Subject: Re: Checking executables before running them in L4re/fiasco On Sat Aug 30, 2014 at 14:02:41 +, Masti Ramya Jayaram wrote: I am using fiasco/l4re to run applications (say hello world) and L4linux. The module list in the two cases looks like this: entry hello-cfg kernel fiasco -serial_esc -freq=533000 roottask moe rom/hello.cfg module l4re module ned module hello.cfg module hello entry l4linux kernel fiasco -serial_esc -freq=533000 roottask moe rom/l4linux.cfg module l4re module ned module l4linux.cfg module io module vmlinuz module ramdisk-x86.rd I would like to add some functionality that performs a few checks on hello and vmlinuz before they are executed. In this context, I have the following questions: a. Where is it best to implement such functionality - moe or ned? Or should I implement a separate server and do this? You can just write your own program for that. In this program, just do normal open()/read()/mmap() on path rom/hello / rom/vmlinuz. b. I need access to an special range of IO memory when I perform the checks. i guess I could always use the io module and do l4io_request_iomem() . Would that be the right way to do it from moe/ned? Yes, just do l4io_request_iomem() to get the IOMEM, however, not from moe nor ned. Ned is just starting io, so you do not have access to io easily from within ned. No problem in a separate program. c. Is there a way to allocate/reserve a dedicated range of physical memory to these applications (hello/vmlinuz) so that they get allocated the same chunk everytime (of physical memory)? You mean for the images? They are placed in memory by the bootloader (and/or bootstrap), so they'll have the same memory location everytime given the bootloader is doing it the same way everytime. They're not moved around afterwards. Adam -- Adam a...@os.inf.tu-dresden.de Lackorzynski http://os.inf.tu-dresden.de/~adam/ ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
New package linking fails
Dear all, I am trying to create a new package under l4/pkg. The linking of the package fails and it has to do with the linker path but I am not sure how to fix it. Here are the details. Package structure: (relevant parts) l4/pkg/test/server/src/main.cc l4/pkg/test/server/src/folder1/test1.c l4/pkg/test/include/test1.h l4/pkg/test/server/src/Makefile PKGDIR ?= ../.. L4DIR ?= $(PKGDIR)/../.. TARGET = $(PKGNAME) # list requirements of your program here REQUIRES_LIBS = libsigma0 libirq l4util l4re_c-util # list your .c or .cc files here SRC_C = folder1/test1.c SRC_CC = main.cc include $(L4DIR)/mk/prog.mk The compilation of the package fails with an undefined reference in main.cc to a function in test1.c (defined through test1.h). All the object files are generated and I checked under build. What do you think is the issue? Thanks, ramya ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
RE: New package linking fails
Okay, I got around it by simply putting the entire header file inclusion in an
extern C{} . Now it works!
Thanks again for your help,
Ramya
From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Masti
Ramya Jayaram [rma...@inf.ethz.ch]
Sent: 01 September 2014 16:53
To: Martin Küttler; l4-hackers@os.inf.tu-dresden.de
Subject: RE: New package linking fails
hey Martin,
I had not done that and I tried it. In main.cc I declared the function as
extern C int self_test(int);
I hope that is what you meant.
However, I get the following error:
test1.h:115: error: previous declaration of ‘int self_test(int)’ with ‘C++’
linkage
/l4/pkg/test/server/src/main.cc:46: error: conflicts with new declaration with
‘C’ linkage
best,
Ramya
From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Martin
Küttler [martin.kuett...@gmail.com]
Sent: 01 September 2014 16:45
To: l4-hackers@os.inf.tu-dresden.de
Subject: Re: New package linking fails
On 09/01/2014 04:28 PM, Masti Ramya Jayaram wrote:
Dear all,
I am trying to create a new package under l4/pkg. The linking of the package
fails and it has to do with the linker path but I am not sure how to fix it.
Here are the details.
Package structure: (relevant parts)
l4/pkg/test/server/src/main.cc
l4/pkg/test/server/src/folder1/test1.c
l4/pkg/test/include/test1.h
l4/pkg/test/server/src/Makefile
PKGDIR ?= ../..
L4DIR ?= $(PKGDIR)/../..
TARGET = $(PKGNAME)
# list requirements of your program here
REQUIRES_LIBS = libsigma0 libirq l4util l4re_c-util
# list your .c or .cc files here
SRC_C = folder1/test1.c
SRC_CC = main.cc
include $(L4DIR)/mk/prog.mk
The compilation of the package fails with an undefined reference in main.cc
to a function in test1.c (defined through test1.h). All the object files are
generated and I checked under build. What do you think is the issue?
Did you declare the function with extern C linkage in main.cc? Without
that the linker won't find C-functions called from C++-files.
Martin
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
Fault in function writev (ulibc)
Dear all, I am running a custom application package on fiasco using a serial line (no keyboard/screen) and I intermittently get faults caused by printf. I notice that on disassembling the code around the error that the function is called writev in ulibc. Has anyone experienced something similar before? Thanks, ramya ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
RE: Minimal requirements to run L4linux using only a serial console
Hi Martin, I replied earlier to the list but I guess I messed it up a bit. Your tip on the polling turned out to be really helpful. I basically changed the type of serial_esc in kernel/fiasco/src/kern/config.cpp from SERIAL_ESC_IRQ to SERIAL_ESC_NOIRQ and it works! It would be great if you can elaborate on the difference (I am guessing the latter uses a poll based system but more explanation would be great). I was even able to run l4 and fiasco on the actual SCC hardware and get the UART RX to work. So thanks a ton for the tip! :) Best, Ramya From: Martin Schröder [mschroe...@web.de] Sent: 29 August 2014 23:23 To: Masti Ramya Jayaram; l4-hackers@os.inf.tu-dresden.de Subject: Re: Minimal requirements to run L4linux using only a serial console Hi Ramya, Tx path of UART is working but the Rx part not due to missing interrupts. In the Tx part interrupts are not really required - alt least not for Hello World. I also don't care about the missing H right now. So the first step would be polling the fiasco UART every 100ms or so instead of waitung for interrupts. Best regards, Martin I don't care abount the missing H, this could be anything. Am 29.08.2014 17:45, Masti Ramya Jayaram schrieb: Hey Martin, No I just realized that ESC does not break into the debugger. But I can clearly see that the process instead of printing Hello world prints ello World :) What could the reason be? Thanks, Ramya From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Martin Schröder [martin.schroe...@openlimit.com] Sent: 29 August 2014 17:35 To: l4-hackers@os.inf.tu-dresden.de Subject: Re: Minimal requirements to run L4linux using only a serial console Hi Ramya, a: No, its not a must. The UART could also be polled (by a timer) to see if something arrived. b: At least regarding l4linux there is nothing more to it - afaik. But let's go back to the hello word example on the SCC. What happens if you press the ESC key? Does it break into the JDB debugger? Best regards, Martin ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
Checking executables before running them in L4re/fiasco
Dear all, I am using fiasco/l4re to run applications (say hello world) and L4linux. The module list in the two cases looks like this: entry hello-cfg kernel fiasco -serial_esc -freq=533000 roottask moe rom/hello.cfg module l4re module ned module hello.cfg module hello entry l4linux kernel fiasco -serial_esc -freq=533000 roottask moe rom/l4linux.cfg module l4re module ned module l4linux.cfg module io module vmlinuz module ramdisk-x86.rd I would like to add some functionality that performs a few checks on hello and vmlinuz before they are executed. In this context, I have the following questions: a. Where is it best to implement such functionality - moe or ned? Or should I implement a separate server and do this? b. I need access to an special range of IO memory when I perform the checks. i guess I could always use the io module and do l4io_request_iomem() . Would that be the right way to do it from moe/ned? c. Is there a way to allocate/reserve a dedicated range of physical memory to these applications (hello/vmlinuz) so that they get allocated the same chunk everytime (of physical memory)? Thanks in advance, ramya ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
RE: Minimal requirements to run L4linux using only a serial console
Hey Martin,
I cleaned up my build and it worked - the mistake was mine. I had a question
regarding ACPI. The target machine I am using has no BIOS/ACPi support. So,
when I boot linux on it my console input does not work.This is similar to
booting in qemu with the -no-acpi option. Serial console output works (I can
see the prompt of L4Linux) but input does not.
Is there something special to do when booting with a serial console but no ACPI
tables?
Thanks,
ramya
From: Martin Schröder [martin.schroe...@openlimit.com]
Sent: 27 August 2014 18:32
To: Masti Ramya Jayaram; l4-hackers@os.inf.tu-dresden.de
Subject: Re: Minimal requirements to run L4linux using only a serial console
Hi Masti,
you have all of them already! I did not alter any other file beside the files
you already sent. I inserted them into a fresh
snapshot
(http://l4re.org/download/snapshots-oc/l4re-snapshot-2014053111.tar.xz) and
started building. Since drops-rd.rd was not
found in the snapshot, I just renamed ramdisk-x86.rd to drops-rd.rd.
Nothing more.
Martin
Am 27.08.2014 um 18:02 schrieb Masti Ramya Jayaram:
Hey Martin,
thanks for the quick response. Could you send me your configs (l4linux,
fiasco, l4re) so that I can try it with that?
Thanks,
Ramya
From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of
Martin Schröder [martin.schroe...@openlimit.com]
Sent: 27 August 2014 16:28
To: l4-hackers@os.inf.tu-dresden.de
Subject: Re: Minimal requirements to run L4linux using only a serial console
Hi Masti,
I just applied your config, replaced drops-rd.rd by ramdisk-x86.rd and
everything (command prompt, debugger) runs fine as
expected in Qemu. Maybe you ran into a similar problem as described in this
thread:
http://os.inf.tu-dresden.de/pipermail/l4-hackers/2012/thread.html#5165
Best regards,
Martin
Am 27.08.2014 um 15:18 schrieb Masti Ramya Jayaram:
Sorry, a couple of typos in the l4linux.cfg
--vim:set ft=lua:
L4.default_loader:start({ caps = {
log = L4.Env.log:m(rws),
},
l4re_dbg = L4.Dbg.Warn,
log = { l4linux, yellow },
}, rom/vmlinuz mem=64M console=ttyLv0 l4x_rd=rom/drops-rd.rd root=1:0
ramdisk_size=4000 l4ser.vkey_enable=1);
Thanks in advance,
ramya
From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of
Masti Ramya Jayaram [rma...@inf.ethz.ch]
Sent: 27 August 2014 14:07
To: l4-hackers@os.inf.tu-dresden.de
Subject: Minimal requirements to run L4linux using only a serial console
Dear all,
I am trying to run L4linux on fiasco/l4re but with the keyboard and VGA
disabled. I can get all the way to the command prompt (of the shell) in the
initramfs but then, on pressing a key, I get a page fault and I cannot debug
the system anymore using the serial_esc. I suspect it is my configuration
which I have provided below. I was wondering what the minimal packages are
to achieve this.
modules.list:
entry l4linux
kernel fiasco -serial_esc -freq=533000
roottask moe rom/l4linux.cfg
module l4re
module ned
module l4linux.cfg
module io
module vmlinuz
module drops-rd.rd
I have disabled the framebuffer and input server in the L4linux stub drivers
configuration and enabled the serial console.
l4linux.cfg:
--vim:set ft=lua:
L4.default_loader:start({ caps = {
log = L4.Env.log:m(rws),
},
l4re_dbg = L4.Dbg.Warn,
log = { l4linux, yellow },
}, rom/io - rom/vmlinuz mem=64M console=ttyLv0
l4x_rd=rom/drops-rd.rd root=1:0 ramdisk$
I also specify a custom frequency for my board to fiasco.oc, do I have to
make similar modifications to l4linux (e.g., Timer frequency)?
Thanks,
Ramya
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
RE: Minimal requirements to run L4linux using only a serial console
Hey Martin,
Back on the list ;-)
Ok, I can confirm that when I busy loop during fiasco init, I can actually see
the character that I typed. So would I have to change the l4ser driver in Linux
or the main driver in under pkg/l4/bootstrap or under fiasco in vkey.cpp? :)
Thanks,
ramya
From: Masti Ramya Jayaram
Sent: 29 August 2014 12:15
To: Martin Schröder; l4-hackers@os.inf.tu-dresden.de
Subject: RE: Minimal requirements to run L4linux using only a serial console
Hey Martin,
I cleaned up my build and it worked - the mistake was mine. I had a question
regarding ACPI. The target machine I am using has no BIOS/ACPi support. So,
when I boot linux on it my console input does not work.This is similar to
booting in qemu with the -no-acpi option. Serial console output works (I can
see the prompt of L4Linux) but input does not.
Is there something special to do when booting with a serial console but no ACPI
tables?
Thanks,
ramya
From: Martin Schröder [martin.schroe...@openlimit.com]
Sent: 27 August 2014 18:32
To: Masti Ramya Jayaram; l4-hackers@os.inf.tu-dresden.de
Subject: Re: Minimal requirements to run L4linux using only a serial console
Hi Masti,
you have all of them already! I did not alter any other file beside the files
you already sent. I inserted them into a fresh
snapshot
(http://l4re.org/download/snapshots-oc/l4re-snapshot-2014053111.tar.xz) and
started building. Since drops-rd.rd was not
found in the snapshot, I just renamed ramdisk-x86.rd to drops-rd.rd.
Nothing more.
Martin
Am 27.08.2014 um 18:02 schrieb Masti Ramya Jayaram:
Hey Martin,
thanks for the quick response. Could you send me your configs (l4linux,
fiasco, l4re) so that I can try it with that?
Thanks,
Ramya
From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of
Martin Schröder [martin.schroe...@openlimit.com]
Sent: 27 August 2014 16:28
To: l4-hackers@os.inf.tu-dresden.de
Subject: Re: Minimal requirements to run L4linux using only a serial console
Hi Masti,
I just applied your config, replaced drops-rd.rd by ramdisk-x86.rd and
everything (command prompt, debugger) runs fine as
expected in Qemu. Maybe you ran into a similar problem as described in this
thread:
http://os.inf.tu-dresden.de/pipermail/l4-hackers/2012/thread.html#5165
Best regards,
Martin
Am 27.08.2014 um 15:18 schrieb Masti Ramya Jayaram:
Sorry, a couple of typos in the l4linux.cfg
--vim:set ft=lua:
L4.default_loader:start({ caps = {
log = L4.Env.log:m(rws),
},
l4re_dbg = L4.Dbg.Warn,
log = { l4linux, yellow },
}, rom/vmlinuz mem=64M console=ttyLv0 l4x_rd=rom/drops-rd.rd root=1:0
ramdisk_size=4000 l4ser.vkey_enable=1);
Thanks in advance,
ramya
From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of
Masti Ramya Jayaram [rma...@inf.ethz.ch]
Sent: 27 August 2014 14:07
To: l4-hackers@os.inf.tu-dresden.de
Subject: Minimal requirements to run L4linux using only a serial console
Dear all,
I am trying to run L4linux on fiasco/l4re but with the keyboard and VGA
disabled. I can get all the way to the command prompt (of the shell) in the
initramfs but then, on pressing a key, I get a page fault and I cannot debug
the system anymore using the serial_esc. I suspect it is my configuration
which I have provided below. I was wondering what the minimal packages are
to achieve this.
modules.list:
entry l4linux
kernel fiasco -serial_esc -freq=533000
roottask moe rom/l4linux.cfg
module l4re
module ned
module l4linux.cfg
module io
module vmlinuz
module drops-rd.rd
I have disabled the framebuffer and input server in the L4linux stub drivers
configuration and enabled the serial console.
l4linux.cfg:
--vim:set ft=lua:
L4.default_loader:start({ caps = {
log = L4.Env.log:m(rws),
},
l4re_dbg = L4.Dbg.Warn,
log = { l4linux, yellow },
}, rom/io - rom/vmlinuz mem=64M console=ttyLv0
l4x_rd=rom/drops-rd.rd root=1:0 ramdisk$
I also specify a custom frequency for my board to fiasco.oc, do I have to
make similar modifications to l4linux (e.g., Timer frequency)?
Thanks,
Ramya
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
RE: Minimal requirements to run L4linux using only a serial console
Also, given that the thing works with ACPI on, wouldn't it be better to include
ACPI information in the boot instead of changing the driver? Or is this
impossible?
Best,
Ramya
From: Masti Ramya Jayaram
Sent: 29 August 2014 15:55
To: Martin Schröder; l4-hackers@os.inf.tu-dresden.de
Subject: RE: Minimal requirements to run L4linux using only a serial console
Hey Martin,
Back on the list ;-)
Ok, I can confirm that when I busy loop during fiasco init, I can actually see
the character that I typed. So would I have to change the l4ser driver in Linux
or the main driver in under pkg/l4/bootstrap or under fiasco in vkey.cpp? :)
Thanks,
ramya
From: Masti Ramya Jayaram
Sent: 29 August 2014 12:15
To: Martin Schröder; l4-hackers@os.inf.tu-dresden.de
Subject: RE: Minimal requirements to run L4linux using only a serial console
Hey Martin,
I cleaned up my build and it worked - the mistake was mine. I had a question
regarding ACPI. The target machine I am using has no BIOS/ACPi support. So,
when I boot linux on it my console input does not work.This is similar to
booting in qemu with the -no-acpi option. Serial console output works (I can
see the prompt of L4Linux) but input does not.
Is there something special to do when booting with a serial console but no ACPI
tables?
Thanks,
ramya
From: Martin Schröder [martin.schroe...@openlimit.com]
Sent: 27 August 2014 18:32
To: Masti Ramya Jayaram; l4-hackers@os.inf.tu-dresden.de
Subject: Re: Minimal requirements to run L4linux using only a serial console
Hi Masti,
you have all of them already! I did not alter any other file beside the files
you already sent. I inserted them into a fresh
snapshot
(http://l4re.org/download/snapshots-oc/l4re-snapshot-2014053111.tar.xz) and
started building. Since drops-rd.rd was not
found in the snapshot, I just renamed ramdisk-x86.rd to drops-rd.rd.
Nothing more.
Martin
Am 27.08.2014 um 18:02 schrieb Masti Ramya Jayaram:
Hey Martin,
thanks for the quick response. Could you send me your configs (l4linux,
fiasco, l4re) so that I can try it with that?
Thanks,
Ramya
From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of
Martin Schröder [martin.schroe...@openlimit.com]
Sent: 27 August 2014 16:28
To: l4-hackers@os.inf.tu-dresden.de
Subject: Re: Minimal requirements to run L4linux using only a serial console
Hi Masti,
I just applied your config, replaced drops-rd.rd by ramdisk-x86.rd and
everything (command prompt, debugger) runs fine as
expected in Qemu. Maybe you ran into a similar problem as described in this
thread:
http://os.inf.tu-dresden.de/pipermail/l4-hackers/2012/thread.html#5165
Best regards,
Martin
Am 27.08.2014 um 15:18 schrieb Masti Ramya Jayaram:
Sorry, a couple of typos in the l4linux.cfg
--vim:set ft=lua:
L4.default_loader:start({ caps = {
log = L4.Env.log:m(rws),
},
l4re_dbg = L4.Dbg.Warn,
log = { l4linux, yellow },
}, rom/vmlinuz mem=64M console=ttyLv0 l4x_rd=rom/drops-rd.rd root=1:0
ramdisk_size=4000 l4ser.vkey_enable=1);
Thanks in advance,
ramya
From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of
Masti Ramya Jayaram [rma...@inf.ethz.ch]
Sent: 27 August 2014 14:07
To: l4-hackers@os.inf.tu-dresden.de
Subject: Minimal requirements to run L4linux using only a serial console
Dear all,
I am trying to run L4linux on fiasco/l4re but with the keyboard and VGA
disabled. I can get all the way to the command prompt (of the shell) in the
initramfs but then, on pressing a key, I get a page fault and I cannot debug
the system anymore using the serial_esc. I suspect it is my configuration
which I have provided below. I was wondering what the minimal packages are
to achieve this.
modules.list:
entry l4linux
kernel fiasco -serial_esc -freq=533000
roottask moe rom/l4linux.cfg
module l4re
module ned
module l4linux.cfg
module io
module vmlinuz
module drops-rd.rd
I have disabled the framebuffer and input server in the L4linux stub drivers
configuration and enabled the serial console.
l4linux.cfg:
--vim:set ft=lua:
L4.default_loader:start({ caps = {
log = L4.Env.log:m(rws),
},
l4re_dbg = L4.Dbg.Warn,
log = { l4linux, yellow },
}, rom/io - rom/vmlinuz mem=64M console=ttyLv0
l4x_rd=rom/drops-rd.rd root=1:0 ramdisk$
I also specify a custom frequency for my board to fiasco.oc, do I have to
make similar modifications to l4linux (e.g., Timer frequency)?
Thanks,
Ramya
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
___
l4-hackers mailing list
l4-hackers
RE: Minimal requirements to run L4linux using only a serial console
Hey Martin, I am trying to run l4linux/fiasco on the Intel SCC which does have a pic-8259. The Intel SCC does not have a BIOS and does not have any ACPI tables which I think qemu generates on the fly. It also does not use a Screen or keyboard. I obtained a patch from Markus Partheymueller which they used to boot fiasco on the SCC. With it I am able to run hello world both in qemu with no-acpi as well as on the SCC successfully. The patch to the best of my knowledge does the following: a. Makes the cmdline also parse an argument so that the frequency is an input (533Mhz) b. Uses the apic timer for scheduling. c. Disables the pic-8259 because it does not exist. Instead the core in the SCC has two interrupt lines and this patch initializes these lines as interrupts to the core. d. Disables keyboard and screen and enables a serial line for communication instead e. Disables super pages because the SCC does not support it. So, when I try the patched fiasco/L4linux on qemu, it works when I do not disable acpi. But since the SCC does not have any ACPI/IO-APIC, when I try to run the same thing on qemu with ACPI off, l4linux boots but is not responsive to input. In short: when I run with I (below) , my l4linux responds to input and with II (below) not. qemu-system-i386 -m 256 -serial stdio -kernel bootstrap_l4linux.elf -append -arg=fiasco,,-serial_esc -freq=533000 qemu-system-i386 -no-acpi -m 256 -serial stdio -kernel bootstrap_l4linux.elf -append -arg=fiasco,,-serial_esc -freq=533000 Now the question is: a. Is the IO-APIC a must for the serial line? b. Are ACPI tables just a matter of writing data at the right memory locations (e..g, qemu lets you give the acpi-table as input I noticed)? Or is there more to it? Thanks for your patience, ramya From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Martin Schröder [martin.schroe...@openlimit.com] Sent: 29 August 2014 16:29 To: l4-hackers@os.inf.tu-dresden.de Subject: Re: Minimal requirements to run L4linux using only a serial console Hi Ramya, Why did you disable the PIC? What is your aim with ACPI and why should this lead to a change in the UART driver? And what has the serial console to do with ACPI tables? For me the topics have at the moment very little in common or I am missing something. Best regards, Martin Am 29.08.2014 um 15:58 schrieb Masti Ramya Jayaram: Also, given that the thing works with ACPI on, wouldn't it be better to include ACPI information in the boot instead of changing the driver? Or is this impossible? Best, Ramya From: Masti Ramya Jayaram Sent: 29 August 2014 15:55 To: Martin Schröder; l4-hackers@os.inf.tu-dresden.de Subject: RE: Minimal requirements to run L4linux using only a serial console Hey Martin, Back on the list ;-) Ok, I can confirm that when I busy loop during fiasco init, I can actually see the character that I typed. So would I have to change the l4ser driver in Linux or the main driver in under pkg/l4/bootstrap or under fiasco in vkey.cpp? :) Thanks, ramya From: Masti Ramya Jayaram Sent: 29 August 2014 12:15 To: Martin Schröder; l4-hackers@os.inf.tu-dresden.de Subject: RE: Minimal requirements to run L4linux using only a serial console Hey Martin, I cleaned up my build and it worked - the mistake was mine. I had a question regarding ACPI. The target machine I am using has no BIOS/ACPi support. So, when I boot linux on it my console input does not work.This is similar to booting in qemu with the -no-acpi option. Serial console output works (I can see the prompt of L4Linux) but input does not. Is there something special to do when booting with a serial console but no ACPI tables? Thanks, ramya From: Martin Schröder [martin.schroe...@openlimit.com] Sent: 27 August 2014 18:32 To: Masti Ramya Jayaram; l4-hackers@os.inf.tu-dresden.de Subject: Re: Minimal requirements to run L4linux using only a serial console Hi Masti, you have all of them already! I did not alter any other file beside the files you already sent. I inserted them into a fresh snapshot (http://l4re.org/download/snapshots-oc/l4re-snapshot-2014053111.tar.xz) and started building. Since drops-rd.rd was not found in the snapshot, I just renamed ramdisk-x86.rd to drops-rd.rd. Nothing more. Martin Am 27.08.2014 um 18:02 schrieb Masti Ramya Jayaram: Hey Martin, thanks for the quick response. Could you send me your configs (l4linux, fiasco, l4re) so that I can try it with that? Thanks, Ramya From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Martin Schröder [martin.schroe...@openlimit.com] Sent: 27 August 2014 16:28 To: l4-hackers@os.inf.tu-dresden.de Subject: Re: Minimal requirements
RE: Minimal requirements to run L4linux using only a serial console
Hey Martin, No I just realized that ESC does not break into the debugger. But I can clearly see that the process instead of printing Hello world prints ello World :) What could the reason be? Thanks, Ramya From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Martin Schröder [martin.schroe...@openlimit.com] Sent: 29 August 2014 17:35 To: l4-hackers@os.inf.tu-dresden.de Subject: Re: Minimal requirements to run L4linux using only a serial console Hi Ramya, a: No, its not a must. The UART could also be polled (by a timer) to see if something arrived. b: At least regarding l4linux there is nothing more to it - afaik. But let's go back to the hello word example on the SCC. What happens if you press the ESC key? Does it break into the JDB debugger? Best regards, Martin Am 29.08.2014 um 16:56 schrieb Masti Ramya Jayaram: Hey Martin, I am trying to run l4linux/fiasco on the Intel SCC which does have a pic-8259. The Intel SCC does not have a BIOS and does not have any ACPI tables which I think qemu generates on the fly. It also does not use a Screen or keyboard. I obtained a patch from Markus Partheymueller which they used to boot fiasco on the SCC. With it I am able to run hello world both in qemu with no-acpi as well as on the SCC successfully. The patch to the best of my knowledge does the following: a. Makes the cmdline also parse an argument so that the frequency is an input (533Mhz) b. Uses the apic timer for scheduling. c. Disables the pic-8259 because it does not exist. Instead the core in the SCC has two interrupt lines and this patch initializes these lines as interrupts to the core. d. Disables keyboard and screen and enables a serial line for communication instead e. Disables super pages because the SCC does not support it. So, when I try the patched fiasco/L4linux on qemu, it works when I do not disable acpi. But since the SCC does not have any ACPI/IO-APIC, when I try to run the same thing on qemu with ACPI off, l4linux boots but is not responsive to input. In short: when I run with I (below) , my l4linux responds to input and with II (below) not. qemu-system-i386 -m 256 -serial stdio -kernel bootstrap_l4linux.elf -append -arg=fiasco,,-serial_esc -freq=533000 qemu-system-i386 -no-acpi -m 256 -serial stdio -kernel bootstrap_l4linux.elf -append -arg=fiasco,,-serial_esc -freq=533000 Now the question is: a. Is the IO-APIC a must for the serial line? b. Are ACPI tables just a matter of writing data at the right memory locations (e..g, qemu lets you give the acpi-table as input I noticed)? Or is there more to it? Thanks for your patience, ramya From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Martin Schröder [martin.schroe...@openlimit.com] Sent: 29 August 2014 16:29 To: l4-hackers@os.inf.tu-dresden.de Subject: Re: Minimal requirements to run L4linux using only a serial console Hi Ramya, Why did you disable the PIC? What is your aim with ACPI and why should this lead to a change in the UART driver? And what has the serial console to do with ACPI tables? For me the topics have at the moment very little in common or I am missing something. Best regards, Martin Am 29.08.2014 um 15:58 schrieb Masti Ramya Jayaram: Also, given that the thing works with ACPI on, wouldn't it be better to include ACPI information in the boot instead of changing the driver? Or is this impossible? Best, Ramya From: Masti Ramya Jayaram Sent: 29 August 2014 15:55 To: Martin Schröder; l4-hackers@os.inf.tu-dresden.de Subject: RE: Minimal requirements to run L4linux using only a serial console Hey Martin, Back on the list ;-) Ok, I can confirm that when I busy loop during fiasco init, I can actually see the character that I typed. So would I have to change the l4ser driver in Linux or the main driver in under pkg/l4/bootstrap or under fiasco in vkey.cpp? :) Thanks, ramya From: Masti Ramya Jayaram Sent: 29 August 2014 12:15 To: Martin Schröder; l4-hackers@os.inf.tu-dresden.de Subject: RE: Minimal requirements to run L4linux using only a serial console Hey Martin, I cleaned up my build and it worked - the mistake was mine. I had a question regarding ACPI. The target machine I am using has no BIOS/ACPi support. So, when I boot linux on it my console input does not work.This is similar to booting in qemu with the -no-acpi option. Serial console output works (I can see the prompt of L4Linux) but input does not. Is there something special to do when booting with a serial console but no ACPI tables? Thanks, ramya From: Martin Schröder [martin.schroe...@openlimit.com] Sent: 27 August 2014 18:32 To: Masti Ramya Jayaram; l4-hackers
RE: Minimal requirements to run L4linux using only a serial console
Also with acpi on, it does enter the debugger in qemu. Best, Ramya From: Masti Ramya Jayaram Sent: 29 August 2014 17:45 To: Martin Schröder; l4-hackers@os.inf.tu-dresden.de Subject: RE: Minimal requirements to run L4linux using only a serial console Hey Martin, No I just realized that ESC does not break into the debugger. But I can clearly see that the process instead of printing Hello world prints ello World :) What could the reason be? Thanks, Ramya From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Martin Schröder [martin.schroe...@openlimit.com] Sent: 29 August 2014 17:35 To: l4-hackers@os.inf.tu-dresden.de Subject: Re: Minimal requirements to run L4linux using only a serial console Hi Ramya, a: No, its not a must. The UART could also be polled (by a timer) to see if something arrived. b: At least regarding l4linux there is nothing more to it - afaik. But let's go back to the hello word example on the SCC. What happens if you press the ESC key? Does it break into the JDB debugger? Best regards, Martin Am 29.08.2014 um 16:56 schrieb Masti Ramya Jayaram: Hey Martin, I am trying to run l4linux/fiasco on the Intel SCC which does have a pic-8259. The Intel SCC does not have a BIOS and does not have any ACPI tables which I think qemu generates on the fly. It also does not use a Screen or keyboard. I obtained a patch from Markus Partheymueller which they used to boot fiasco on the SCC. With it I am able to run hello world both in qemu with no-acpi as well as on the SCC successfully. The patch to the best of my knowledge does the following: a. Makes the cmdline also parse an argument so that the frequency is an input (533Mhz) b. Uses the apic timer for scheduling. c. Disables the pic-8259 because it does not exist. Instead the core in the SCC has two interrupt lines and this patch initializes these lines as interrupts to the core. d. Disables keyboard and screen and enables a serial line for communication instead e. Disables super pages because the SCC does not support it. So, when I try the patched fiasco/L4linux on qemu, it works when I do not disable acpi. But since the SCC does not have any ACPI/IO-APIC, when I try to run the same thing on qemu with ACPI off, l4linux boots but is not responsive to input. In short: when I run with I (below) , my l4linux responds to input and with II (below) not. qemu-system-i386 -m 256 -serial stdio -kernel bootstrap_l4linux.elf -append -arg=fiasco,,-serial_esc -freq=533000 qemu-system-i386 -no-acpi -m 256 -serial stdio -kernel bootstrap_l4linux.elf -append -arg=fiasco,,-serial_esc -freq=533000 Now the question is: a. Is the IO-APIC a must for the serial line? b. Are ACPI tables just a matter of writing data at the right memory locations (e..g, qemu lets you give the acpi-table as input I noticed)? Or is there more to it? Thanks for your patience, ramya From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Martin Schröder [martin.schroe...@openlimit.com] Sent: 29 August 2014 16:29 To: l4-hackers@os.inf.tu-dresden.de Subject: Re: Minimal requirements to run L4linux using only a serial console Hi Ramya, Why did you disable the PIC? What is your aim with ACPI and why should this lead to a change in the UART driver? And what has the serial console to do with ACPI tables? For me the topics have at the moment very little in common or I am missing something. Best regards, Martin Am 29.08.2014 um 15:58 schrieb Masti Ramya Jayaram: Also, given that the thing works with ACPI on, wouldn't it be better to include ACPI information in the boot instead of changing the driver? Or is this impossible? Best, Ramya From: Masti Ramya Jayaram Sent: 29 August 2014 15:55 To: Martin Schröder; l4-hackers@os.inf.tu-dresden.de Subject: RE: Minimal requirements to run L4linux using only a serial console Hey Martin, Back on the list ;-) Ok, I can confirm that when I busy loop during fiasco init, I can actually see the character that I typed. So would I have to change the l4ser driver in Linux or the main driver in under pkg/l4/bootstrap or under fiasco in vkey.cpp? :) Thanks, ramya From: Masti Ramya Jayaram Sent: 29 August 2014 12:15 To: Martin Schröder; l4-hackers@os.inf.tu-dresden.de Subject: RE: Minimal requirements to run L4linux using only a serial console Hey Martin, I cleaned up my build and it worked - the mistake was mine. I had a question regarding ACPI. The target machine I am using has no BIOS/ACPi support. So, when I boot linux on it my console input does not work.This is similar to booting in qemu with the -no-acpi option. Serial console output works (I can see the prompt of L4Linux
RE: Minimal requirements to run L4linux using only a serial console
Hi Martin, That was a very useful tip. I simply changed the default value in kernel/fiasco/src/kern/config.cpp from SERIAL_ESC_IRQ to SERIAL_ESC_NOIRQ which leads to polling (I am guessing) and voila! It works! I even tried on the actual SCC and it works! I can break into the debugger as well as input to l4linux. Thanks a lot for your help for your help. Best, Ramya From: Masti Ramya Jayaram Sent: 29 August 2014 17:48 To: Martin Schröder; l4-hackers@os.inf.tu-dresden.de Subject: RE: Minimal requirements to run L4linux using only a serial console Also with acpi on, it does enter the debugger in qemu. Best, Ramya From: Masti Ramya Jayaram Sent: 29 August 2014 17:45 To: Martin Schröder; l4-hackers@os.inf.tu-dresden.de Subject: RE: Minimal requirements to run L4linux using only a serial console Hey Martin, No I just realized that ESC does not break into the debugger. But I can clearly see that the process instead of printing Hello world prints ello World :) What could the reason be? Thanks, Ramya From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Martin Schröder [martin.schroe...@openlimit.com] Sent: 29 August 2014 17:35 To: l4-hackers@os.inf.tu-dresden.de Subject: Re: Minimal requirements to run L4linux using only a serial console Hi Ramya, a: No, its not a must. The UART could also be polled (by a timer) to see if something arrived. b: At least regarding l4linux there is nothing more to it - afaik. But let's go back to the hello word example on the SCC. What happens if you press the ESC key? Does it break into the JDB debugger? Best regards, Martin Am 29.08.2014 um 16:56 schrieb Masti Ramya Jayaram: Hey Martin, I am trying to run l4linux/fiasco on the Intel SCC which does have a pic-8259. The Intel SCC does not have a BIOS and does not have any ACPI tables which I think qemu generates on the fly. It also does not use a Screen or keyboard. I obtained a patch from Markus Partheymueller which they used to boot fiasco on the SCC. With it I am able to run hello world both in qemu with no-acpi as well as on the SCC successfully. The patch to the best of my knowledge does the following: a. Makes the cmdline also parse an argument so that the frequency is an input (533Mhz) b. Uses the apic timer for scheduling. c. Disables the pic-8259 because it does not exist. Instead the core in the SCC has two interrupt lines and this patch initializes these lines as interrupts to the core. d. Disables keyboard and screen and enables a serial line for communication instead e. Disables super pages because the SCC does not support it. So, when I try the patched fiasco/L4linux on qemu, it works when I do not disable acpi. But since the SCC does not have any ACPI/IO-APIC, when I try to run the same thing on qemu with ACPI off, l4linux boots but is not responsive to input. In short: when I run with I (below) , my l4linux responds to input and with II (below) not. qemu-system-i386 -m 256 -serial stdio -kernel bootstrap_l4linux.elf -append -arg=fiasco,,-serial_esc -freq=533000 qemu-system-i386 -no-acpi -m 256 -serial stdio -kernel bootstrap_l4linux.elf -append -arg=fiasco,,-serial_esc -freq=533000 Now the question is: a. Is the IO-APIC a must for the serial line? b. Are ACPI tables just a matter of writing data at the right memory locations (e..g, qemu lets you give the acpi-table as input I noticed)? Or is there more to it? Thanks for your patience, ramya From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Martin Schröder [martin.schroe...@openlimit.com] Sent: 29 August 2014 16:29 To: l4-hackers@os.inf.tu-dresden.de Subject: Re: Minimal requirements to run L4linux using only a serial console Hi Ramya, Why did you disable the PIC? What is your aim with ACPI and why should this lead to a change in the UART driver? And what has the serial console to do with ACPI tables? For me the topics have at the moment very little in common or I am missing something. Best regards, Martin Am 29.08.2014 um 15:58 schrieb Masti Ramya Jayaram: Also, given that the thing works with ACPI on, wouldn't it be better to include ACPI information in the boot instead of changing the driver? Or is this impossible? Best, Ramya From: Masti Ramya Jayaram Sent: 29 August 2014 15:55 To: Martin Schröder; l4-hackers@os.inf.tu-dresden.de Subject: RE: Minimal requirements to run L4linux using only a serial console Hey Martin, Back on the list ;-) Ok, I can confirm that when I busy loop during fiasco init, I can actually see the character that I typed. So would I have to change the l4ser driver in Linux or the main driver in under pkg/l4/bootstrap or under fiasco in vkey.cpp
Minimal requirements to run L4linux using only a serial console
Dear all,
I am trying to run L4linux on fiasco/l4re but with the keyboard and VGA
disabled. I can get all the way to the command prompt (of the shell) in the
initramfs but then, on pressing a key, I get a page fault and I cannot debug
the system anymore using the serial_esc. I suspect it is my configuration which
I have provided below. I was wondering what the minimal packages are to achieve
this.
modules.list:
entry l4linux
kernel fiasco -serial_esc -freq=533000
roottask moe rom/l4linux.cfg
module l4re
module ned
module l4linux.cfg
module io
module vmlinuz
module drops-rd.rd
I have disabled the framebuffer and input server in the L4linux stub drivers
configuration and enabled the serial console.
l4linux.cfg:
--vim:set ft=lua:
L4.default_loader:start({ caps = {
log = L4.Env.log:m(rws),
},
l4re_dbg = L4.Dbg.Warn,
log = { l4linux, yellow },
}, rom/io - rom/vmlinuz mem=64M console=ttyLv0 l4x_rd=rom/drops-rd.rd
root=1:0 ramdisk$
I also specify a custom frequency for my board to fiasco.oc, do I have to make
similar modifications to l4linux (e.g., Timer frequency)?
Thanks,
Ramya
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
RE: Minimal requirements to run L4linux using only a serial console
Hey Martin,
thanks for the quick response. Could you send me your configs (l4linux, fiasco,
l4re) so that I can try it with that?
Thanks,
Ramya
From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Martin
Schröder [martin.schroe...@openlimit.com]
Sent: 27 August 2014 16:28
To: l4-hackers@os.inf.tu-dresden.de
Subject: Re: Minimal requirements to run L4linux using only a serial console
Hi Masti,
I just applied your config, replaced drops-rd.rd by ramdisk-x86.rd and
everything (command prompt, debugger) runs fine as
expected in Qemu. Maybe you ran into a similar problem as described in this
thread:
http://os.inf.tu-dresden.de/pipermail/l4-hackers/2012/thread.html#5165
Best regards,
Martin
Am 27.08.2014 um 15:18 schrieb Masti Ramya Jayaram:
Sorry, a couple of typos in the l4linux.cfg
--vim:set ft=lua:
L4.default_loader:start({ caps = {
log = L4.Env.log:m(rws),
},
l4re_dbg = L4.Dbg.Warn,
log = { l4linux, yellow },
}, rom/vmlinuz mem=64M console=ttyLv0 l4x_rd=rom/drops-rd.rd root=1:0
ramdisk_size=4000 l4ser.vkey_enable=1);
Thanks in advance,
ramya
From: l4-hackers [l4-hackers-boun...@os.inf.tu-dresden.de] on behalf of Masti
Ramya Jayaram [rma...@inf.ethz.ch]
Sent: 27 August 2014 14:07
To: l4-hackers@os.inf.tu-dresden.de
Subject: Minimal requirements to run L4linux using only a serial console
Dear all,
I am trying to run L4linux on fiasco/l4re but with the keyboard and VGA
disabled. I can get all the way to the command prompt (of the shell) in the
initramfs but then, on pressing a key, I get a page fault and I cannot debug
the system anymore using the serial_esc. I suspect it is my configuration
which I have provided below. I was wondering what the minimal packages are to
achieve this.
modules.list:
entry l4linux
kernel fiasco -serial_esc -freq=533000
roottask moe rom/l4linux.cfg
module l4re
module ned
module l4linux.cfg
module io
module vmlinuz
module drops-rd.rd
I have disabled the framebuffer and input server in the L4linux stub drivers
configuration and enabled the serial console.
l4linux.cfg:
--vim:set ft=lua:
L4.default_loader:start({ caps = {
log = L4.Env.log:m(rws),
},
l4re_dbg = L4.Dbg.Warn,
log = { l4linux, yellow },
}, rom/io - rom/vmlinuz mem=64M console=ttyLv0 l4x_rd=rom/drops-rd.rd
root=1:0 ramdisk$
I also specify a custom frequency for my board to fiasco.oc, do I have to
make similar modifications to l4linux (e.g., Timer frequency)?
Thanks,
Ramya
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
___
l4-hackers mailing list
l4-hackers@os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
Fiasco, L4Re, L4linux for Pentium 5
Dear all, I am trying to run L4linux on top of fiasco/L4re using an older version from the svn (for those interested, I am trying to run this on Intel SCC and therefore need an old build). I am first trying to run this on qemu and can get the entire linux to boot and display a shell prompt. However, I get a pagefault on trying to enter a character on the shell prompt as below: ... Switching to clocksource tsc VFS: Mounted root (ext2 filesystem) on device 1:0. Doing post... Freeing unused kernel memory: 164k freed # l4linux | cpu0: utcb=0xb3000600 416 l4linux | l4lx_thread_create: Created thread 423 (idler0) (u:b3000c00, sp:006a1ff8) l4linux | Page fault: addr = 03f2 pc = 004f34dc (rw) l4linux | Page fault (non-resolved): pfa=3f2 pc=4f34dc ... I can verify that the first character typed at the prompt reaches the fiasco kernel (because I can echo it). I am not sure how exactly this (tries to) reach L4linux from there on. Note that I have VGA and keyboards disabled and I am trying to use only the serial line for input and output. Has anyone had similar problems before? Any help on how to proceed would be great! best, ramya ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
Re: L4-based microkernel on Intel SCC
Markus Partheymueller mpartheym at os.inf.tu-dresden.de writes: The changes to the kernel have not been merged into upstream because they were very specific to the SCC and of little more generic use. But I can send you the patch if you want. That would be great! Also, do let me know the version of the compiler, fiasco and l4re you used just to be sure. Alternatively, the source tree will also work. Yes, you should be able to run almost every ELF binary (conditions may apply...) produced by the L4Re build system (make elfimage E=yourmoduleentry), including L4Linux. But I have to say that we did not have the time for looking into larger deployments at the time. In order to load and start ELF binaries on the SCC, I wrote an extension to the sccKit that was available then. The way it worked was like this: 1) Extract ELF according to the ELF headers and copy it to the SCC's memory 2) Place protected mode boot code and the reset vector in memory 3) Set up multiboot information structure (for memory maps) 4) Set up an I/O connection for providing serial connection with the core 1,2,3 are possible but 4 depends on the OS I guess. Send me this as well in any case and I will get back to you if I have problems. Now, if I remember correctly, a lot of improvement has been made to the sccKit and the whole interaction with the chip after I finished this project (virtual framebuffers, serial output, etc.). So I tend to doubt that this part of my (fairly outdated) work will be of any practical use to you. But you could of course have a look at it. No problem. I will get back to you if I have problems. Thanks, Ramya ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
L4-based microkernel on Intel SCC
Hi all, In fact, I am looking for a micro-kernel based hypervisor (like L4:pistachio or Fiasco) to run on the SCC. The only pointer I found was a port of Fiasco OS for Intel SCC recently (http://os.inf.tu-dresden.de/papers_ps/partheymueller-beleg.pdf) and would like to try this. I had a few questions though: a. Is the public version of fiasco compatible with the SCC? If not, is it possible to get it? b. Since SCC uses a different compiler set (much older than gcc 4.4), does the compilation work out of the box? c. Finally, has anyone run L4Linux on top of the SCC? Thanks in advance, Ramya ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
Re: l4re compilation
Adam Lackorzynski adam at os.inf.tu-dresden.de writes: Yes, please stick with 4.8 for now. Adam Thanks Adam! I will try that then. ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
l4re compilation
Hi all, I am trying to compile l4re run time from the snapshot (l4re-snapshot- 2014053111) using GCC 4.9 on an x86-64 machine with multilib. The compilation fails with not being able to find cstdio header file. I followed the instructions in http://os.inf.tu-dresden.de/L4Re/build.html. I note that the fiasco kernel itself builds fine from the same snapshot. Some debug information: Output of gcc -v Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/lto-wrapper Target: x86_64-unknown-linux-gnu Configured with: /build/gcc-multilib/src/gcc-4.9-20140507/configure -- prefix=/usr --libdir=/usr/lib --libexecdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=https://bugs.archlinux.org/ -- enable-languages=c,c++,ada,fortran,go,lto,objc,obj-c++ --enable-shared -- enable-threads=posix --with-system-zlib --enable-__cxa_atexit --disable- libunwind-exceptions --enable-clocale=gnu --disable-libstdcxx-pch --disable- libssp --enable-gnu-unique-object --enable-linker-build-id --enable-cloog- backend=isl --disable-cloog-version-check --enable-lto --enable-plugin -- enable-install-libiberty --with-linker-hash-style=gnu --enable-multilib -- disable-werror --enable-checking=release Thread model: posix gcc version 4.9.0 20140507 (prerelease) (GCC) -- Output of make O=build in the src/l4 directory (of what I think maybe relevant parts) === Building package libstdc++-headers === bash: line 0: cd: /home/ramya/Downloads/fiasco/l4re-snapshot- 2014053111/src/l4/pkg/libstdc++-headers/../libstdc++-v3/contrib/libstdc++- v3-/include: No such file or directory ... ... ... ... ... ... === Building package cxx_libc_io === ... Compiling io_backend.o /home/ramya/Downloads/fiasco/l4re-snapshot- 2014053111/src/l4/pkg/cxx_libc_io/lib/src/io_backend.cc:9:18: fatal error: cstdio: No such file or directory #include cstdio ^ compilation terminated. Is this a problem with the GCC version? Any help is much appreciated. Best, Ramya ___ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
Re: [Wikipl-l] projekt namawiania do uwolnienia zdjęć
jak najbardziej dostępny https://commons.wikimedia.org/wiki/Category:Files_from_Korea.net_Flickr_stream masti On 03/28/2014 01:31 PM, anw...@o2.pl wrote: witam,niestety link Korea Płd już dziś niedostępny Dnia 26 marca 2014 22:32 masti lt;mast...@gmail.comgt; napisał(a): Nasz kolega z Australii russavia prowadzi projekt namawiania użytkowników serwisu Flickr do uwolnienia swoich zdjęć tam załadowanych i opublikowania ich na licencji CC-BY-SA. Projekt dotyczy głównie instytucji rządowych i firm mających swoje konta na Flickrze. Jeśli macie namiary na konta takich użytkowników przekażcie je proszę uzytkownikowi russavia na Commons: https://commons.wikimedia.org/wiki/User:Russavia lub mailowo na adres gmailowy Wzory wiadomości do użytkowników Flickra: https://commons.wikimedia.org/wiki/User:Russavia/Flickr-letter/pl Lista już zidentyfikowanych dostępna jest tu: lt;https://docs.google.com/spreadsheet/ccc?key=0Asj_w_yHF5ymdEREZVJzOGlYR3pTX2dCeGpGOGZtNFEamp;usp=drive_web#gid=0gt; Przykładem skuteczności działania jest konto rządu Korei Płd., który zmienił licencję już następnego dnia po zgłoszeniu prośby udostępniając na wolnej locencji kilka tysię zdjęć: lt;https://commons.wikimedia.org/wiki/Category:Files_from_Korea.net_Flickr_streamgt; masti ___ Lista dyskusyjna WikiPL-l WikiPL-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikipl-l ___ Lista dyskusyjna WikiPL-l WikiPL-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikipl-l ___ Lista dyskusyjna WikiPL-l WikiPL-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikipl-l
[Wikipl-l] projekt namawiania do uwolnienia zdjęć
Nasz kolega z Australii russavia prowadzi projekt namawiania użytkowników serwisu Flickr do uwolnienia swoich zdjęć tam załadowanych i opublikowania ich na licencji CC-BY-SA. Projekt dotyczy głównie instytucji rządowych i firm mających swoje konta na Flickrze. Jeśli macie namiary na konta takich użytkowników przekażcie je proszę uzytkownikowi russavia na Commons: https://commons.wikimedia.org/wiki/User:Russavia lub mailowo na adres gmailowy Wzory wiadomości do użytkowników Flickra: https://commons.wikimedia.org/wiki/User:Russavia/Flickr-letter/pl Lista już zidentyfikowanych dostępna jest tu: https://docs.google.com/spreadsheet/ccc?key=0Asj_w_yHF5ymdEREZVJzOGlYR3pTX2dCeGpGOGZtNFEusp=drive_web#gid=0 Przykładem skuteczności działania jest konto rządu Korei Płd., który zmienił licencję już następnego dnia po zgłoszeniu prośby udostępniając na wolnej locencji kilka tysię zdjęć: https://commons.wikimedia.org/wiki/Category:Files_from_Korea.net_Flickr_stream masti ___ Lista dyskusyjna WikiPL-l WikiPL-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikipl-l
Re: [Imports] Fwd: [osm-pl] Import adresów Gmina Wieprz
On 02/21/2014 01:54 PM, Serge Wroclawski wrote: On Thu, Feb 20, 2014 at 7:48 PM, Jason Remillard remillard.ja...@gmail.com mailto:remillard.ja...@gmail.com wrote: Hello Everybody, Each municipality is a different import. Part of the process needs to be to be validation, for example, which has already been brought up and not answered. I have been trying to follow this thread, but I think I missed this part. Could somebody recap this, if this one really one huge import across all of Poland, or a bunch of little imports. This is a bunch of little imports. This is not a single all of Poland dataset, but rather a collection of little datasets by region. The data is similar, of course due to the nature of the data being addresses. this is true. Data comes form local governaments databases/GIS which are build according to common ministerial rules. masti ___ Imports mailing list Imports@openstreetmap.org https://lists.openstreetmap.org/listinfo/imports
Re: EventAdmin Bundle Installation
Thanks Marcel for the quick response. Can someone help clarify: Can I use EventAdmin to pass the event data from an non-osgi bundle to an osgi bundle? Or is the eventAdmin only used for communicating between osgi bundles? My main application receiving the metrics data/event is non-osgi and it needs to forward this event/data to an osgi-bundle. I was using this before: bundleContext.registerService(SOME_DATA_TYPE, dataHolderMap, null); This works fine for low volume of calls but starts to hang the thread gets blocked. Thanks Masti On Saturday, February 1, 2014 2:44 PM, Marcel Offermans marcel.offerm...@luminis.eu wrote: On 01 Feb 2014, at 23:20 pm, masti whoknows masti...@yahoo.com wrote: Here is simple Java code that I am using to install the EventAdmin bundle //now install the EventAdmin: org.apache.felix.eventadmin 1.3.2 Bundle eventAdminBundle = framework.getBundleContext().installBundle(org.apache.felix.eventadmin); eventAdminBundle.start();Is this not the right way? No, as you can see from your own stacktrace, installBundle takes a location [1] which is interpreted as a URL. Alternatively, you can use the other installBundle method [2] that takes a location (which can then be anything) and an inputstream (to your bundle). That should at least get you to the next point. :) To learn more about embedding Felix, please read [3] and especially the Host/Felix interaction part of that document. It contains a lot of information related to what you're trying to do. Greetings, Marcel [1] http://www.osgi.org/javadoc/r4v43/core/org/osgi/framework/BundleContext.html#installBundle(java.lang.String) [2] http://www.osgi.org/javadoc/r4v43/core/org/osgi/framework/BundleContext.html#installBundle(java.lang.String, java.io.InputStream) [3] http://felix.apache.org/documentation/subprojects/apache-felix-framework/apache-felix-framework-launching-and-embedding.html
Re: EventAdmin Bundle Installation
Neil you are everywhere my Friend! It is good to see people willing to help
others!
You got the understanding of the application correctly. My Application installs
bundles and all bundles live on single JVM.
I am able to install the felix org.apache.felix.eventadmin with version
1.3.2 bundle.
Now pushing the events using the following example:
http://felix.apache.org/site/apache-felix-event-admin.html is not clear with as
how the casting will work.
if (ref != null) { EventAdmin eventAdmin = (EventAdmin)
context.getService(ref);
Here EventAdmin should be casted to what? The only class/package available from
felix is impl package and that does not have EventAdmin.
If I use import org.osgi.service.event.Event;import
org.osgi.service.event.EventAdmin; which I see also available through the same
felix bundle I get the following class cast
exceptionjava.lang.ClassCastException:
org.apache.felix.eventadmin.impl.security.EventAdminSecurityDecorator
incompatible with org.osgi.service.event.EventAdmin
I am sure I am doing something wrong or missing something.
Any pointers are appreciated
Thanks
Masti
On Sunday, February 2, 2014 8:30 AM, Neil Bartlett njbartl...@gmail.com wrote:
On Sun, Feb 2, 2014 at 4:16 PM, masti whoknows masti...@yahoo.com wrote:
Thanks Marcel for the quick response.
Can someone help clarify: Can I use EventAdmin to pass the event data from an
non-osgi bundle to an osgi bundle? Or is the eventAdmin only used for
communicating between osgi bundles?
Yes Event Admin can be used in a non-OSGi environment. However it is not a
remote or distributed event broker, it works purely within a single JVM
(although some people have built remoting gateways or bridges that work
alongside Event Admin).
Therefore if you're talking about using it with both non-OSGi and OSGi
artifacts then you probably mean you have an embedded OSGi framework in your
application. Assuming that's the case, then the embedding application can
interact with Event Admin by publishing an EventHandler interface into OSGi
using the system bundle's BundleContext. The limitation is that the API package
for Event Admin will need to be exported by the system bundle using the
FRAMEWORK_SYSTEM_PACKAGES_EXTRA property.
My main application receiving the metrics data/event is non-osgi and it needs
to forward this event/data to an osgi-bundle. I was using this before:
bundleContext.registerService(SOME_DATA_TYPE, dataHolderMap, null);
Ah yes... I recall your StackOverflow question.
This works fine for low volume of calls but starts to hang the thread gets
blocked.
Thanks
Masti
On Saturday, February 1, 2014 2:44 PM, Marcel Offermans
marcel.offerm...@luminis.eu wrote:
On 01 Feb 2014, at 23:20 pm, masti whoknows masti...@yahoo.com wrote:
Here is simple Java code that I am using to install the EventAdmin bundle
//now install the EventAdmin: org.apache.felix.eventadmin 1.3.2
Bundle eventAdminBundle =
framework.getBundleContext().installBundle(org.apache.felix.eventadmin);
eventAdminBundle.start();Is this not the right way?
No, as you can see from your own stacktrace, installBundle takes a location
[1] which is interpreted as a URL. Alternatively, you can use the other
installBundle method [2] that takes a location (which can then be anything)
and an inputstream (to your bundle).
That should at least get you to the next point. :)
To learn more about embedding Felix, please read [3] and especially the
Host/Felix interaction part of that document. It contains a lot of
information related to what you're trying to do.
Greetings, Marcel
[1]
http://www.osgi.org/javadoc/r4v43/core/org/osgi/framework/BundleContext.html#installBundle(java.lang.String)
[2]
http://www.osgi.org/javadoc/r4v43/core/org/osgi/framework/BundleContext.html#installBundle(java.lang.String,
java.io.InputStream)
[3]
http://felix.apache.org/documentation/subprojects/apache-felix-framework/apache-felix-framework-launching-and-embedding.html
EventAdmin Bundle Installation
Hi OSGi Gurus, I am new to OSGi. I have an application that processes messages/metrics events in a high volume My main application which recieves these events/metrics is an non-OSGi bundle. However, the task for this application is to forward the event/payload to an OSGi bundle. I don't have issues with OSGi bundle as I am able to install the bundle and also added the EventHandler there.My issue occurs when I send the events from my main application. My service reference is always null. In short EventAdmin reference is always null. I am sure I need to install the EventAdmin as a bundle. How can I do it at the top level application? Can you please help me the actual Java code/pseudo that would install the bundle? Dependencies in my pom: dependencies dependency groupIdcom.ebay.external/groupId artifactIducirrus-db/artifactId version0.7.3/version /dependency dependency groupIdorg.apache.felix/groupId artifactIdorg.apache.felix.framework/artifactId version4.2.1/version /dependency dependency groupIdorg.apache.felix/groupId artifactIdorg.apache.felix.bundlerepository/artifactId version1.6.6/version /dependency dependency groupIdorg.ops4j.pax.url/groupId artifactIdpax-url-mvn/artifactId version1.3.6/version /dependency dependency groupIdorg.apache.felix/groupId artifactIdorg.apache.felix.eventadmin/artifactId version1.3.2/version /dependency dependency groupIdmockit/groupId artifactIdjmockit/artifactId version0.999.4/version typejar/type scopetest/scope /dependency dependency groupIdjunit/groupId artifactIdjunit/artifactId scopetest/scope version4.5/version!--$NO-MVN-MAN-VER$ -- /dependency dependency groupIdorg.osgi/groupId artifactIdorg.osgi.core/artifactId version4.3.0/version!--$NO-MVN-MAN-VER$ -- typejar/type scopecompile/scope /dependency dependency groupIdorg.osgi/groupId artifactIdorg.osgi.compendium/artifactId version4.3.1/version typejar/type scopecompile/scope /dependency /dependencies Here is simple Java code that I am using to install the EventAdmin bundle //now install the EventAdmin: org.apache.felix.eventadmin 1.3.2 Bundle eventAdminBundle = framework.getBundleContext().installBundle(org.apache.felix.eventadmin); eventAdminBundle.start();Is this not the right way? Any pointers are highly appreciated Thanks Masti
Re: [Wikipl-l] Wikipedyści w radiu i TV
czyli jednak PR robi podcasty z audycji a naszej w 4 porach roku nie ma :( masti On 01/15/2014 10:17 PM, Lantuszka . wrote: hej, podkast z audycji, w ktorej bralam udzial dostepny jest tu: http://www.polskieradio.pl/130/2351/Artykul/1025399,Wiki-zamiast-Britanniki Lantuszka W dniu 15 stycznia 2014 19:55 użytkownik Dorożyński Janusz (4-webd) dorozyns...@wampnm.webd.pl napisał: | -Original Message- | From: wikipl-l-boun...@lists.wikimedia.org [mailto:wikipl-l- | boun...@lists.wikimedia.org] On Behalf Of Adam Kliczek | Sent: Wednesday, January 15, 2014 4:22 PM / | Na stronie TVP Warszawa jest już dostępne nagranie programu Zapraszamy na | kawę, w którym wystąpiłem: | | http://regionalna.tvp.pl/13657621/15012014-1430 | | Niestety, na początku trzeba obejrzeć około minuty reklam. :/ Ale warto przetrzymać tę minutę. Adamie, dobre wypadłeś. Na upartego można by wyłapać jeden czy dwa lapsusy, ale to już tylko z perspektywy mamuta (takiego jak ja :-)) ). Z uszanowaniem, Janusz Ency Dorożyński ___ Lista dyskusyjna WikiPL-l WikiPL-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikipl-l ___ Lista dyskusyjna WikiPL-l WikiPL-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikipl-l ___ Lista dyskusyjna WikiPL-l WikiPL-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikipl-l
[Wikipl-l] Podcaster i Masti w Polskim Radio
dodam tez info, że o 9:00 będziemy z Borysem w audycji 4 Pory Roku w Polskim Radio Program I http://www.polskieradio.pl/7/163/Artykul/1023928,15-stycznia-Sroda-w-Czterech-Porach-Roku masti ___ Lista dyskusyjna WikiPL-l WikiPL-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikipl-l
