Freenet unstable testing prerelease 1468-pre3 is now available. It is more progress toward 1468. Compared to 1468-pre2 it primarily contains updated builds of official plugins. Library, Spider, and WebOfTrust should now function. The two remaining plugins are FlogHelper, which needs updated backup and restore support, and ThawIndexBrowser, which needs build and NPE fixes.
To use this prerelease either shut down your node and replace
freenet.jar (or whatever is in wrapper.conf classpath), or run
update.(sh|cmd) testing
to download it over clearnet. Inserts are pending and I will reply with
keys once they are complete.
Fred changes since 1468-pre2:
---
Bert Massop (5):
SplitFileInserterStorage: provide linear upper bounds on chooseBlock
SplitFileInserterStorage: adapt chooseBlock to coding standards
SplitFileInserterStorage: separate random iteration logic
Add unit tests for RandomArrayIterator
SplitFileFetcherStorage: use RandomArrayIterator for segment choice
Matthew Toseland (1):
Fix bucket leak on non-persistent inserts.
Steve Dougherty (8):
RandomArrayIteratorTest: add missing EOF newline
Update Spider to v52
Mark XMLLibrarian and XMLSpider as unsupported
Increase Spider minimum version
Update Library to v36
Update WebOfTrust to build0015
Fix whitespace and braces
Use consistent capitalization in bookmark l10n keys
Update translations
TheSeeker (2):
Fix #6513 Implement text list of inserted keys.
Fix: Transfer backoff was never being reset.
drak@kaverne (1):
default bookmarks: add my sharewiki and update descriptions
Library v36
---
Linus Tolke (1):
Avoid the use of the "freenet:"-prefix.
Matthew Toseland (4):
Add two new indexes, woot!
Fix build for purge-db4o merge.
v36: Cleanups, inspired by TheSeeker's post-purge. (Not Yaml updates!)
Fix leak
Steve Dougherty (1):
Update build for broken snakeyaml link
Spider v52
---
David ‘Bombe’ Roden (1):
Include time of last activity in web interface.
Matthew Toseland (4):
Fix priority
Add missing @Override's
Use Spider as RequestClient, don't need a new one.
No, we don't want to use the local freenet root URI. Explain why.
TheSeeker (1):
Try to make Spider work post-purge.
WebOfTrust build0014 / build0015
---
This build features a very large amount of bug fixes and code quality
and test improvements. We hope these improvements will speed future
development.
Even though the primary purpose of this build is to alleviate the
performance issues, it will still be very slow until applications built
on top of WOT have been adapted to use the new performance features.
The main new performance improvement is the so-called
"event-notifications" FCP API which enables apps to request notification
of changes without having to download the whole WOT database every few
minutes. This cannot be put to use without applications being changed to
use it.
If you are a developer of a WOT application, please read the "Changelog
for developers" section for instructions how to wire it in to your app.
There are at least 5 further strong algorithmic performance
optimizations possible within WOT itself which I shall work on for the
following releases. Those then should speed up WOT without requiring
adaptions from client application developers. It was chosen to implement
event-notifications before these so we can have a parallelization of
development: Client app developers can start integrating
event-notifications now, while I can work on the WOT core optimizations.
What follows now are two changelogs of this build:
- One of the new features which are primarily relevant to the users
- One for new stuff which can be used by developers of applications
which use WOT.
The changelogs are briefly sorted by importance, most important stuff
first. The number left to each change is the issue number in the
bugtracker. Code quality improvements and some bug fixes are not
included here. For the unfiltered list see
https://bugs.freenetproject.org/changelog_page.php?version_id=63
Changelog for users:
- 0006039: [Web Interface] WebInterface "Log in" refactoring TODOs (xor)
Operhiem1 helped a lot with dealing with the job of refactoring the
general style of the web interface to allow users to "Log in" with a
certain identity instead of having to chose which identity to work
with at every page. This should greatly increase usability as it
matches the general convention of how websites work.
Also, this functionality can be used by authors of other plugins so
there is one common WOT login for anything which uses WOT.
- 0006027: [Web Interface] Backport LogInPage features from Freetalk to
WOT (xor)
The new "Log in" page features a wizard for creating and restoring
identities. This wizard explains what WOT is, what an identity is,
what trust values are, etc. It allows the user to chose preferences.
Also, it fixes the issue of people mixing up the features for
restoring an existing identity and creating a new one from a SSK,
which could have caused their identity to be permanently broken. This
is fixed by explaining the difference very clearly.
Same as with the "Log in" page, the wizard can be used by other
plugins.
- 0006225: [Web Interface] Make the KnownIdentitiesPage split the list
into pages (xor)
Previously, the "Known identities" page would list ALL identites in a
single page. This was obviously not acceptable anymore since we have
> 11 000 identities nowadays.
Now the list is split into pages.
- [Not in bugtracker] [Web Interface]: Translation updates
Add French and Simplified Chinese translations, and update Italian,
Dutch, and Russian.
Thanks to the volunteers on Transifex, and thanks to operhiem1 for
putting this into a pull request.
Notice: This is probably incomplete already because it was done before
adding the "Create identity" wizard. AFAIK, the new stuff is up on
Transifex already, so if you want to contribute with translating it,
please contact me or operhiem1. Mail x...@freenetproject.org, or even
better join the #freenet channel on Freenode IRC.
- [No bugtracker entry] [Web Interface]: Tell the user whats invalid
bout a chosen nickname
When the user enters a nick which contains non-allowed characters, we
now tell him which those characters are.
- 0006136: [Web Interface] "Restore identity" is only accessible if the
user creates an identity first (xor)
Previously, the menu for restoring an old identity was only accessible
if you actually had already created least one identity. The
"Restore identity" feature can now be used through the
"Create identity" wizard without creating a completely new identity
first.
- 0006148: [Web Interface] Randomly generated nicknames may exceed 30
character limit (xor)
The nicknames now always fit the 30 character limit.
- 0005354: [Web Interface] Usability of the web interface (xor)
When clicking on "Community" or "Community -> Home", the "Own
anonymous identities" page is now shown instead of the "Statistics"
page.
- 0006458: [Performance] Parallelize shutdown (xor)
During shutdown, instead of waiting for each subsystem of WOT to have
terminated before termining the next one, we now terminate them all in
parallel using threads.
This should speed up shutdown a lot (but please don't expect wonders,
there are further issues with shutdown which the next release will
improve).
- 0006447: [Performance] IdentityInserter thread period constant is too
low and causes inserts to never succeed (xor)
This fixed bug would prevent your identities from ever being inserted
unless you had very high uptime.
- 0006001: [Core] Restore WebOfTrust.db4o.backup if WebOfTrust.db4o does
not exist (xor)
This fixed bug would happen if your computer crashed while WOT was
starting up. The symptom would be loss of your database, i.e. all your
identities would be gone.
- 0004854: [Web Interface] Stacktrace when clicking 'switch to simple
mode' while viewing an identity (xor)
Clicking this web interface link of Freenet did cause WOT to display
an error, which is now fixed.
- [No bugtracker entry] [Performancde] New seed identity, thanks to
operhiem1
Having another seed identity should speed up the bootstrapping for new
users.
Changelog for developers:
- [Various bugtracker entries] Event-notifications:
This new FCP API should provide a HUGE reduction of the load which
client apps put onto WOT.
What this changes is as follows: To get an up-to-date list of all
known identities (or trusts/scores), you previously had to download
ALL of them by FCP every X minutes. You would do this even if NONE of
them changed - a huge CPU/IO load for nothing. Now you just
"Subscribe" to the list of identities. As a result, WOT will send you
all of them once as you subscribe, and then keep you up to date on its
own by sending changed identities only when a change happens.
Thus, instead of a constant load of O(N = number of identites) every X
minutes, independent of the amount of changes, you now get a load of
O(1) for each change as it happens.
If you are a client application developer, and want to integrate
event-notifications in your app, I strongly recommend you to
1) Use the new classes DebugFCPClient and
FCPClientReferenceImplementation. The latter aims at being
something which you can copy-paste to your client application
plugin so you have a fully working FCP client which you can use to
connect to WOT and obtain event-notifications. The DebugFCPClient
uses the aforementioned reference FCP client to have WOT connect
to itself and test whether event-notifications ships correct
information (test runs of this have been done for several nights).
For your purposes, it can also be considered as an example of how
to actually use the FCPClientReferenceImplementation.
2) Contact me in case you need help in understanding how to use it! :)
Mail x...@freenetproject.org, or even better join the #freenet
channel on Freenode IRC.
- 0006039: [Web Interface] WebInterface "Log in" refactoring TODOs (xor)
(Even though this seems something for users, it IS interesting for
developers, please continue reading)
Operhiem1 helped a lot with dealing with the job of refactoring the
general style of the web interface to allow users to "Log in" with a
certain identity instead of having to chose which identity to work
with at every page.
And this functionality can be used by your plugins! You can redirect
users of your plugin to the log in page of WOT, and have WOT redirect
the users back to your plugin after they have logged in. For an
example of how to use this, see
https://github.com/Thynix/plugin-DVCS-WebUI/blob/4893cb22d919c0d6b9178711dbb4990f66a52232/src/main/java/org/freenetproject/plugin/dvcs_webui/ui/web/VelocityToadlet.java
Please contact me or operhiem1 for further help. Mail
x...@freenetproject.org, or even better join the #freenet channel on
Freenode IRC.
- 0006027: [Web Interface] Backport LogInPage features from Freetalk to
WOT (xor)
The new "Log in" page features a wizard for creating and restoring
identities.
Same as with the "Log in" page, the wizard can be used by other
plugins. For an example of how to use this, contact me. Mail
x...@freenetproject.org, or even better join the #freenet channel on
Freenode IRC.
- 0006231: [Core] Finish synchronous FCP API at fred (xor)
The implementation of event-notification spawned a huge 569-commit
side-project at Freenet itself aka "plugin-fcp-rewrite": The rewrite
of the FCP API for communication with plugins, and especially for
plugin-to-plugin FCP communication.
This adds a lot of new features to replace the code duplication in
existing plugins, as they had usually implemented their own extensions
of the old API to fix its shortcomings. It also provides new stuff
which was completely impossible with the previous API. Notably for
example the ability for servers to send messages to the client on
their own, without any recent message from the client. This allows
plugin FCP server authors to deploy events to clients.
Also it improves the code quality of the FCP plugin API a lot - the
old API was completely undocumented, the new one has full JavaDoc.
There is a much longer and more complete description of whats new in
the new API at the pull request. Anyone who has written a plugin which
offers an FCP server or uses FCP to connect to another plugin such as
WOT, please read it: https://github.com/freenet/fred/pull/319
Notice: WOT will for a while stay backwards-compatible to users of the
old API. However, to use event-notifications, you must use the new
API. I would strongly recommend you to just copypaste the WOT class
FCPClientReferenceImplementation for communicating with WOT by FCP, it
does use the new API alredy.
- [Pull request 26] Background jobs
Bertm did a great job with implementing a framework for replacing
dangerous usage of the fred implementations of interface Ticker such
as especially TrivialTicker. This might be interesting for fred code
and other plugins which use those fred classes. The issue was that the
existing fred classes provide a "no duplicates" flag which sounds like
it ensures that a scheduled job will only ever be executed once. But
under certain conditions, the job can actually be executed multiple
times in parallel. The new interface "BackgroundJob" and its
implementations provide a way to run a thread with a time delay and
the guarantee that it will execute only once in parallel. This is a
candidate for being moved to fred or a common library.
Bertm saved me probably weeks of work with that, thanks a lot!
- [No bugtracker entry]: Full-node unit tests
WOT now contains class "AbstractFullNodeTest" which can be used as a
base class for unit tests. This class starts a regular Freenet node,
and loads the WOT plugin into it just as it would be loaded during
normal operation of Freenet. From a quality point of view, this is a
lot better since it provides a more real environment than the previous
approach of manually creating certain WOT classes in a synthetic
environment. Also, it allows unit tests to use the full capabilities
of Freenet. This is a foundation for future tests which could do fancy
stuff such as testing the Identity introduction code by doing real
uploads / downloads of puzzles and their solutions.
Notice: This might also be interesting to authors of other plugins.
The code is pretty generic and should be easy to re-use there.
- [No bugtracker entry]: Randomized unit tests which aim to cover all
code paths of WOT
Function doRandomChangesToWOT() in the new class
AbstractJUnit4BaseTest is a framework for thorough, randomized unit
tests. It simulates real usage by doing a random amount of changes
chosen among many of the possible UI action:
- WebOfTrust.createOwnIdentity()
- WebOfTrust.deleteOwnIdentity()
- WebOfTrust.restoreOwnIdentity()
- WebOfTrust.restoreOwnIdentity() with existing colliding non-own
Identity
- WebOfTrust.addIdentity()
- WebOfTrust.addContext() (adds context to identity)
- WebOfTrust.setProperty (adds property to identity)
- Add/change/Remove trust values
This can help with testing stuff for which it is difficult to say what
might cause problems.
It is already used for testing-event notifications: Any of the
mentioned user actions are sources of events, so it is a good idea to
test a random permutation of them for whether event-notifications
ships their events correctly.
Thanks to:
- operhiem1 for various pull requests, the great amount of web
interface work, and insane amount of code review.
- bertm for the giant background-job pull request which probably saved
me weeks worth of work
- ArneBab for various pull requests and code review
- TheSeeker for testing
- anyone who I forgot about
- and basically EVERYONE in the project for having the great great
patience of awating this release for over a year. Sorry again that
it took so long! The only explanation I can offer of why this
happend is because I wanted to not release half-finished stuff
anymore given that the whole WOT peformance disaster was a result of
deploying half-finished code as an official plugin. Nevertheless, I
hope to have learned something from all the complaints about the
huge delay of the release, and will try to have a MUCH shorter
release cycle in the future.
signature.asc
Description: OpenPGP digital signature
_______________________________________________ Devl mailing list Devl@freenetproject.org https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
