Re: Numbered List of Freenet Project Ideas

2019-07-21 Thread yanmaani
1.  Introduce a restricted plugin API: nextgens has had this idea for a 
while -
only give plugins a plugin API object instead of a full classloader 
with access
to the entire Node. If plugins can only access Fred functions 
explicitly
intended for plugin use, it should be way easier to figure out how 
plugins are
intended to do things, way easier to document and find documentation 
for

because it’s all in one area, and it could also have better security
characteristics with more work - like a system for plugins to be 
granted and
restricted from various capabilities, both within Freenet’s API and on 
the
system. It’d be better for backwards compatibility going forward 
because now
it’s defined instead of plugins reaching into the Node object and 
poking at

whatever they found to make things work.

2.  Make FCP into a plugin: bonus points if it uses the restricted 
plugin API.
It’d be a good way to test the plugin API and ensure it covers a good 
amount of
functionality. This would separate an important function from core Fred 
code.
(Side note: FCP really should require two-sided user interaction like 
KeepassXC
and its browser companion plugins, or an API key, instead of giving 
access to
whatever plugin can connect to its port.) The plugin might also be able 
to be
run in a passive mode in a transition phase, like GitHub can do with 
changes it
makes, where its answer to a query is compared to that of the 
integrated FCP.
(This’d have to be for some subset of queries that doesn’t change the 
state of
things - are there many/any of those? Maybe fall back to integrated FCP 
for

things not implemented by the plugin?)


Maybe it would be a good idea to use a platform-neutral API? Then people 
could develop their plugins in whatever languages they wanted, instead 
of having to tightly couple them to Java. So either you'd connect over 
TCP to a port, or you'd use some kind of C API, which nearly every 
language trivially can make bindings for.


For smaller plugins, FCP could execute them, like in CGI. It would be 
some work to sandbox them though. One way around it would be to not have 
the system execute them, and for instance restrict yourself to Java at 
first, but make it possible to extend to other language. Personally, I'm 
a stickler for Lua, and that sandboxes very well.


As for queries changing the internal state, the nuclear option would be 
to provide some "testing mode" where you run two instances and replicate 
everything. But then what about the non-deterministic parts?


Another approach might be to gradually change FCP to interface with a 
replica of the plugin API but still be integrated non-plugin functions 
anymore, it can be separated out cleanly. In a second step, the plugin 
API could be changed into a Java binding for a C plugin API, which in 
turn is implemented as a C binding for a Java plugin API.


4.  Transport plugins: we need more things Freenet traffic can look 
like and
more ways for it to move around, and as a bonus doing this will likely 
continue

to turn up bugs in various places.

What about Shoeshop? Could that architecture be extended?


Not to mention a Tor Hidden
Service transport plugin - listen as a hidden service, and connect to 
other

nodes so listening.
Isn't I2P better for such applications? Freenet does use a lot of 
bandwidth, after all.


Another low-hanging fruit would be BitTorrent. The DHT could be used as 
a backup seed node. BitTorrent traffic doesn't draw attention, is 
already encrypted, and doesn't look odd connecting to specific sets of 
IPs. The GFW does use this heuristic to detect proxies - if a certain 
percent of outbound bandwidth goes to an IP, then that gets blocked.


6.  Can we adapt existing applications?: forgetting for a moment how 
JavaScript
adverse much of our userbase is, can those who are willing to run it 
(not from
Freesites but from local applications) be served by writing a backend 
powered
by Freenet for an existing forum or wiki? How practical would that be 
to do?
https://github.com/Requarks/wiki  
https://github.com/discourse/discourse I feel
like we’re spending a lot of effort trying to write these applications 
from the
ground up, and while that does have some desirable properties it’s also 
more

work for fewer features.


For what purpose? Wikis are entirely static already, and to engineer a 
forum does more or less require a from-the-ground-up re-architecture 
anyway. And such an engineering work has been undertaken already, see 
FMS.


I wouldn't think it's possible, anyway. Most of these applications rely 
on secret databases and incoming connections, of which Freenet furnishes 
for neither. You'd just be using Freenet as a poor man's Tor, as I 
understand it.


Then you'd be better off running conventional MediaWiki software, using 
a script to dump all of the pages as HTML to a directory, and then 
publishing it as a conventional freesite. The downside is of course the 
read-only nature and the poor scaling. Be

Numbered List of Freenet Project Ideas

2019-05-11 Thread Steve Dougherty
Hi there. Wall of text incoming.

Here are some projects which I think would take Freenet in some interesting 
directions:

Capabilities / APIs
---

1.  Introduce a restricted plugin API: nextgens has had this idea for a while - 
only give plugins a plugin API object instead of a full classloader with access 
to the entire Node. If plugins can only access Fred functions explicitly 
intended for plugin use, it should be way easier to figure out how plugins are 
intended to do things, way easier to document and find documentation for 
because it’s all in one area, and it could also have better security 
characteristics with more work - like a system for plugins to be granted and 
restricted from various capabilities, both within Freenet’s API and on the 
system. It’d be better for backwards compatibility going forward because now 
it’s defined instead of plugins reaching into the Node object and poking at 
whatever they found to make things work.

2.  Make FCP into a plugin: bonus points if it uses the restricted plugin API. 
It’d be a good way to test the plugin API and ensure it covers a good amount of 
functionality. This would separate an important function from core Fred code. 
(Side note: FCP really should require two-sided user interaction like KeepassXC 
and its browser companion plugins, or an API key, instead of giving access to 
whatever plugin can connect to its port.) The plugin might also be able to be 
run in a passive mode in a transition phase, like GitHub can do with changes it 
makes, where its answer to a query is compared to that of the integrated FCP. 
(This’d have to be for some subset of queries that doesn’t change the state of 
things - are there many/any of those? Maybe fall back to integrated FCP for 
things not implemented by the plugin?)

3.  Supersede FCP with Protocol Buffers and gRPC: it may have made sense at the 
time, but now there are well-established methods of defining network protocols 
and RPCs, and we’re really not doing ourselves favors by continuing to roll our 
own. Doing something like this would make it easier to build applications that 
use Freenet, like a mobile app that connects to a node.

4.  Transport plugins: we need more things Freenet traffic can look like and 
more ways for it to move around, and as a bonus doing this will likely continue 
to turn up bugs in various places. There’s a PR for this from GSoC but it has 
long since stalled. https://github.com/ValveSoftware/GameNetworkingSockets 
would be very cool to layer Freenet on top of. Not to mention a Tor Hidden 
Service transport plugin - listen as a hidden service, and connect to other 
nodes so listening.

5.  Protocol Buffers for network communication: similar thoughts to FCP using 
protocol buffers for local node control. It’d remove weird custom code and make 
alternate implementations easier. (There’s still problem of using Reijendal but 
that’s separate and seems more manageable.)

6.  Can we adapt existing applications?: forgetting for a moment how JavaScript 
adverse much of our userbase is, can those who are willing to run it (not from 
Freesites but from local applications) be served by writing a backend powered 
by Freenet for an existing forum or wiki? How practical would that be to do? 
https://github.com/Requarks/wiki  https://github.com/discourse/discourse I feel 
like we’re spending a lot of effort trying to write these applications from the 
ground up, and while that does have some desirable properties it’s also more 
work for fewer features.


Packaging
-

1.  First party distro packages: this may be easier once we’re finally done 
splitting freenet-ext. I’d been working on it but didn’t manage to finish so 
far. https://github.com/Thynix/fred-staging/tree/remove-freenet-ext I’m not 
opposed to standalone packaging in principle, but using package managers would 
probably both remove the need to do a bunch of careful update work ourselves 
and allow better distro-specific integration and adaptation.

2.  Better support for package repositories in Freenet: I think this has been 
discussed previously, and the Mempo folks ran up against the lack of this 
pretty hard, but one way to do it would be to have a MirrorUSK plugin that can 
mirror the contents of a USK/manifest to disk. Then, distro packages can 
install Fred with this MirrorUSK plugin already installed and configured to 
mirror their repo, and point the package manager at the directory on disk 
unless requested not to. Tada! Distro-specific updates over Freenet.

- Steve