Re: [dev] [surf] Webkit2 with proxy server

[hiro]

10x smaller is certainly not suckless either. check the archives, you
only need a couple lines to parse the youtube bullshit for the real
video file with some sed or so.

On Sun, Dec 25, 2016 at 6:00 AM, Ivan Tham  wrote:
>
> On Sat, Dec 24, 2016 at 10:03:51PM +, Teodoro Santoni wrote:
>>
>> Hi Laslo,
>>
>> 2016-12-24 10:11 GMT, Laslo Hunhold :

 Anything else can be solved by finding your way into scraping the
 website and building a proxy that sends you a very simplified version
 of it at your w3m, links, lynx, dillo, mosaic or shell script.
 It isn't easier and should be packed into a standard distribution,
 but it isn't all that much of an enigma.
>>>
>>>
>>> Never go full Stallman.
>>
>>
>> I'm just lazy, like very lazy, and the youtube-dl model looks appealing
>> to me: trusting someone dedicated to a naked representation
>> of a single website, not doing the scraper yourself... I although know
>> well that a package manager or a single standard distrib leads
>> to drawbacks like dbus, keith packard and the funny story of lpad
>> on npm (node devs deserved it in full!) [0].
>
>
>> youtube-dl:  A small command-line program to download videos from
>> YouTube.com and a few more sites
>
>
> And don't forget that there is you-get which is 10x smaller than
> youtube-dl. Youtube-dl isn't considered small.
>
>
> --
> Do what you like, like what you do.  -- Pickfire
>

Re: [dev] [surf] Webkit2 with proxy server

[sylvain . bertrand]

On Sun, Dec 25, 2016 at 01:00:20PM +0800, Ivan Tham wrote:
> 
> On Sat, Dec 24, 2016 at 10:03:51PM +, Teodoro Santoni wrote:
> > Hi Laslo,
> > 
> > 2016-12-24 10:11 GMT, Laslo Hunhold :
> > > > Anything else can be solved by finding your way into scraping the
> > > > website and building a proxy that sends you a very simplified version
> > > > of it at your w3m, links, lynx, dillo, mosaic or shell script.
> > > > It isn't easier and should be packed into a standard distribution,
> > > > but it isn't all that much of an enigma.
> > > 
> > > Never go full Stallman.
> > 
> > I'm just lazy, like very lazy, and the youtube-dl model looks appealing
> > to me: trusting someone dedicated to a naked representation
> > of a single website, not doing the scraper yourself... I although know
> > well that a package manager or a single standard distrib leads
> > to drawbacks like dbus, keith packard and the funny story of lpad
> > on npm (node devs deserved it in full!) [0].
> 
> > youtube-dl:  A small command-line program to download videos from
> > YouTube.com and a few more sites
> 
> And don't forget that there is you-get which is 10x smaller than
> youtube-dl. Youtube-dl isn't considered small.

Anything python is certainly _not_ suckless, or we'll end up like any
mainstream distro: to get the "full" experience for the "end-user",
7328748239473892473289 different scripting engines are _required_ (python2,
python3, javascript, perl, guile, swift, ruby, lua.). That is not suckless,
this is a pile of fat slimy shit.

(and the SDKs for devs are just disgusting, starting with the autotools, which
are the products of sick brains).

-- 
Sylvain

Re: [dev] [surf] Webkit2 with proxy server

[Ivan Tham]

On Sat, Dec 24, 2016 at 10:03:51PM +, Teodoro Santoni wrote:

Hi Laslo,

2016-12-24 10:11 GMT, Laslo Hunhold :

Anything else can be solved by finding your way into scraping the
website and building a proxy that sends you a very simplified version
of it at your w3m, links, lynx, dillo, mosaic or shell script.
It isn't easier and should be packed into a standard distribution,
but it isn't all that much of an enigma.


Never go full Stallman.


I'm just lazy, like very lazy, and the youtube-dl model looks appealing
to me: trusting someone dedicated to a naked representation
of a single website, not doing the scraper yourself... I although know
well that a package manager or a single standard distrib leads
to drawbacks like dbus, keith packard and the funny story of lpad
on npm (node devs deserved it in full!) [0].



youtube-dl:  A small command-line program to download videos from
YouTube.com and a few more sites


And don't forget that there is you-get which is 10x smaller than
youtube-dl. Youtube-dl isn't considered small.

--
Do what you like, like what you do.  -- Pickfire

Re: [dev] [surf] Webkit2 with proxy server

[Teodoro Santoni]

Hi Laslo,

2016-12-24 10:11 GMT, Laslo Hunhold :
>> Anything else can be solved by finding your way into scraping the
>> website and building a proxy that sends you a very simplified version
>> of it at your w3m, links, lynx, dillo, mosaic or shell script.
>> It isn't easier and should be packed into a standard distribution,
>> but it isn't all that much of an enigma.
>
> Never go full Stallman.

I'm just lazy, like very lazy, and the youtube-dl model looks appealing
to me: trusting someone dedicated to a naked representation
of a single website, not doing the scraper yourself... I although know
well that a package manager or a single standard distrib leads
to drawbacks like dbus, keith packard and the funny story of lpad
on npm (node devs deserved it in full!) [0].

[0] http://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/

Re: [dev] [surf] Webkit2 with proxy server

[sylvain . bertrand]

On Sat, Dec 24, 2016 at 11:11:56AM +0100, Laslo Hunhold wrote:
> > Anything else can be solved by finding your way into scraping the
> > website and building a proxy that sends you a very simplified version
> > of it at your w3m, links, lynx, dillo, mosaic or shell script.
> > It isn't easier and should be packed into a standard distribution,
> > but it isn't all that much of an enigma.

The problem _is_ the web devs.

I know some people are going to dislike this, but amazon does show the right 
way:
http://m.amazon.com

Shopping and payment, noscript from begining to end. Works nicely with lynx:
Fair Web.

Any critical www site (for instance online payment and bank account sites)
which does not provide similar portal should be sued for lack of interop. State
and administration related www sites should not even be allowed to exist
without such a www portal.

Regarding youtube: they should provide a noscript portal. They just need to put
their dash manifest in the  and/or  markup. With the proper mime
type we'll pass it to a player able to deal with the dash manifest (just need
to keep dash manifest complexity and stability in check).  If you read their
condition of usage: YOU MUST USE THE JAVASCRIPT PLAYER TO PLAY VIDEO.  (we know
the code is obfuscated on purpose for DRM...). I guess they can be sued on this
for lack of interop.

-- 
Sylvain

Re: [dev] [surf] Webkit2 with proxy server

[sylvain . bertrand]

On Fri, Dec 23, 2016 at 11:49:15PM +, Josuah Demangeon wrote:
> On 2016-12-17 20:22, Cág wrote:
> > And this is why the web engine should be in C, hackable, simple and
> > small enough
> > to be compiled with tcc/pcc/scc, just like other suckless software
> > (I haven't tried scc though, looks like active work is in progress by
> > glancing
> > at the log and hackers list).
> 
> I just found this engine [1], it seems to have a better approach, even if I
> did not run it yet.  It is not a finished software, but at least there are
> ongoing efforts to write a C99 HTML renderer without dependencies.
> 
> [1]: https://lexborisov.github.io/Modest/

He's solo. Netsurf dev_s_ are unable to propose an enabled javascript www
renderer: because it's massive and the complexity is insane (something about
thousands of "hooks" related to the dynamicity of the renderer).

-- 
Sylvain

Re: [dev] [surf] Webkit2 with proxy server

[Laslo Hunhold]

On Sat, 24 Dec 2016 01:17:51 +
Teodoro Santoni  wrote:

Hey Teodoro,

> Obfuscation of the data used to draw the web application is also
> often done on purpose, to protect the website's revenue from
> adblockers. I dispute the idea that web services like Soundcloud
> would need an interface like the one Soundcloud has right now, unless
> it needs to have a player-omnibar-comment interface.

I totally agree with you. It also explains, why YouTube changes so
often, in order to break third-party tools like youtube-dl.
Admittedly, the youtube-dl developers define the word masochism, which
in a good way shows that with enough dedication you can really create
something wonderful.
And not only with YouTube, youtube-dl also saves you from the mess that
twitch is.

> But it's not that grim like I painted it, the only two real problems
> are
> * with vital-tier websites like online banking impeding to avoid to
> be burdened by the choice of either the single preferred brand and
> model of web browser or the mobile app on a smartphone, avoiding the
> PC entirely;
> * with static document archives which are switching to https, which
> is good when you're reading a blog of whatever, but is bad when you
> have to browse your township's website after a major catastrophe, for
> example.

Yes, especially the forced TLS bugs me often. Call me old-fashioned,
but sometimes, websites force TLS and only accept new, shiny curves
that have been published for a few months.
It goes so far that some websites don't even allow you to connect if
your browser/TLS-stack is older than a year.
And for pages like lobste.rs, which does it in a less harsh way but
still, I don't see the reason to connect with TLS enforced.

> Anything else can be solved by finding your way into scraping the
> website and building a proxy that sends you a very simplified version
> of it at your w3m, links, lynx, dillo, mosaic or shell script.
> It isn't easier and should be packed into a standard distribution,
> but it isn't all that much of an enigma.

Never go full Stallman.

Cheers

Laslo

-- 
Laslo Hunhold 

Re: [dev] [surf] Webkit2 with proxy server

[Josuah Demangeon]

On 2016-12-17 20:22, Cág wrote:
And this is why the web engine should be in C, hackable, simple and 
small enough

to be compiled with tcc/pcc/scc, just like other suckless software
(I haven't tried scc though, looks like active work is in progress by 
glancing

at the log and hackers list).


I just found this engine [1], it seems to have a better approach, even 
if I did not run it yet.  It is not a finished software, but at least 
there are ongoing efforts to write a C99 HTML renderer without 
dependencies.


[1]: https://lexborisov.github.io/Modest/

Re: [dev] [surf] Webkit2 with proxy server

[Sylvain BERTRAND]

On Sat, Dec 17, 2016 at 09:08:39PM +0100, Martin Kühne wrote:
> Talking about banks who make things inherently complicated, I heard
> some people use gnucash and similar things to do their bank stuff,
> though I don't know how much functionality is there effectively
> available on this stuff. I just know my own bank which is otherwise
> really preferable for being Swiss doesn't offer use of an "open"
> interface.

Main online banking use case: a table to see the log of operations on your
accounts.

You do not need javascript for that with basic web standards: basic web forms
are already enough to support online end-user banking.

-- 
Sylvain

Re: [dev] [surf] Webkit2 with proxy server

[Sylvain BERTRAND]

On Sun, Dec 18, 2016 at 03:32:25PM +, Cág wrote:
> Sylvain BERTRAND wrote:
> 
> >Logs? Are you talking about a javascript enabled dynamic www renderer
> >or C compilers?
> 
> scc logs, here at suckless.

scc/qbe good things... but no happy end since they won't be able to one-step
compile a modern c++ compiler for many critical components.

>>Because, people at google screwe* us: they made gcc c++.
> 
> Not really, they allowed C++ in the compiler itself but not that
> much. C compiler is still in C. Check it out yourself here:
> https://github.com/gcc-mirror/gcc/find/master
> Type .cc or .cpp, you'll see it's not much.

You must have a c++98 compiler. A modern gcc will _refuse_ to configure and
compile.

I plan to attempt a C port of a modern c++ gcc, that in order to keep the
number of bootstrap steps as low as possible.

-- 
Sylvain

Re: [dev] [surf] Webkit2 with proxy server

[Sylvain BERTRAND]

On Sun, Dec 18, 2016 at 04:15:10PM +0100, Laslo Hunhold wrote:
> However, we still have bootstrapping, so gcc might as well be written in
> Brainfuck. As long as you can bootstrap it, everything is alright.

You cannot bootstrap with C in a reasonable way.

For instance you want to compile that pile of cra* which is llvm (those 
_amazing_
coders got it wrong right at the start, LOL) for GPU shader support. It's 
advanced
c++ and it does not compile with gcc 4.7.4, the last C bootstrapable compiler.
It means you have to compile first c++ gcc 4.7.4 in order to compile c++ gcc 
6.0.2,
that to compile llvm.

I acquired a raspberry pi 3... armv8/64bits... I was happy and about to
configure a minimal server... till I found out that gcc armv8 backend (aarch64)
is *NOT* in gcc 4.7.4
So in order to bootstrap from a C compiler linux for armv8, I must compile c++ 
gcc
4.7.4 on x86/x86_64 in order to compile a cross 6.0.2 C gcc compiler for armv8. 
WOW!

Back on llvm: you are supposed to need "only" a c++98 compiler/runtime to build
llvm... and gcc 4.7.4 pretends to be c++98 clean... of course since llvm is
pushing hard on c++, you run into c++98 front-end gcc 4.7.4 bugs... fixed in
the _first_ c++98 gcc, namely gcc 4.8.0.

Those c++ coders are just fu**ing the open source software stack and are a
bunch of retarded sickoz thinking they are smart.

And writting a C compiler won't really help: modern c++ compiles only with c++.
Don't tell me to compile gcc 4.7.4, look at this bloody mess.

The only way out from this sabotage would to port all c++ written critical 
software
in simple C. For a lambda end-user system, you can just forget it:
 - libreoffice
 - llvm
 - blink/webkit/cef | gecko
 - harfbuzz
Most of them are massive. Maybe writting a modern c++ in simple C in order to
break the vicious cycle? A modern c++ compiler/runtime is just insanity and
ultra hard to get right (have a look at the c++ ABI to have a good laught).

I can hear people LOLing in microsoft and apple.
All this is becoming a joke, a real fu**ing joke.

-- 
Sylvain

Re: [dev] [surf] Webkit2 with proxy server

[Cág]

Sylvain BERTRAND wrote:


Logs? Are you talking about a javascript enabled dynamic www renderer
or C compilers?


scc logs, here at suckless.


Because, people at google screwe* us: they made gcc c++.


Not really, they allowed C++ in the compiler itself but not that
much. C compiler is still in C. Check it out yourself here:
https://github.com/gcc-mirror/gcc/find/master
Type .cc or .cpp, you'll see it's not much.

It's means that the reasonable starting point of any modern distros is 
a

c++ compiler and runtime! In order to compile linux and many critical
applications (like for those who want a modern www renderer).


The dependence on GCC is stupid, right, just like on any other GNU 
software.
The Linux kernel can't be compiled with anything other than this or 
Clang,

which is in C++, but at least licensed under BSD or something.

Cág

Re: [dev] [surf] Webkit2 with proxy server

[Laslo Hunhold]

On Sun, 18 Dec 2016 16:02:39 +0100
Sylvain BERTRAND  wrote:

Hey Sylvain,

> Because, people at google screwe* us: they made gcc c++. It's means
> that the reasonable starting point of any modern distros is a c++
> compiler and runtime! In order to compile linux and many critical
> applications (like for those who want a modern www renderer).
> 
> A side effect: it's making any C compiler kind of useless for lambda
> end-user needs. (You must have a c++ compiler to compile a modern c++
> compiler not a C compiler).

I'd much rather like to see gcc being written in C instead of C++. As
with any GNU project, the codebase is a big mess and the developers
drank the C++-cool-aid instead of fixing things.
Most people don't know how to design proper data structures, so they
end up "loving" templates and overloaded operators in C++ because it
seems like an "elegant" approach to their faulty idea of data
structures.

However, we still have bootstrapping, so gcc might as well be written in
Brainfuck. As long as you can bootstrap it, everything is alright.

Cheers

Laslo

-- 
Laslo Hunhold 

Re: [dev] [surf] Webkit2 with proxy server

[Sylvain BERTRAND]

On Sat, Dec 17, 2016 at 08:22:02PM +, Cág wrote:
> to be compiled with tcc/pcc/scc, just like other suckless software
> (I haven't tried scc though, looks like active work is in progress by
> glancing at the log and hackers list).

Logs? Are you talking about a javascript enabled dynamic www renderer or C 
compilers?

Because, people at google screwe* us: they made gcc c++. It's means that the
reasonable starting point of any modern distros is a c++ compiler and runtime!
In order to compile linux and many critical applications (like for those who
want a modern www renderer).

A side effect: it's making any C compiler kind of useless for lambda end-user 
needs.
(You must have a c++ compiler to compile a modern c++ compiler not a C 
compiler).

-- 
Sylvain

Re: [dev] [surf] Webkit2 with proxy server

[Cág]

Sylvain BERTRAND wrote:

c++ software pulls also a c++ compiler and runtime in their 
dependencies, which

are horrendous compared to a C compiler/miminal runtime.


And this is why the web engine should be in C, hackable, simple and 
small enough

to be compiled with tcc/pcc/scc, just like other suckless software
(I haven't tried scc though, looks like active work is in progress by 
glancing

at the log and hackers list).


Moreover that design will be so fuc*ed up, it will
generate tons of bugs and security holes to justify a good salary for 
idiotic

bosses (or as corrupted as their coders).


"The more lines you write, the more $$$ you earn"

I'm about to sue some french administrations over their www sites and 
my french

bank www sites for their lack of interop with light www browsers.


Is there actually a law to support your suit?

Some say Flash/JS should (have been) disappear (-ed) with HTML5, but I 
see no signs

unfortunately.

Cág

Re: [dev] [surf] Webkit2 with proxy server

[hiro]

no, it's great they all are incompetent to create usable apis, it
means i don't have to waste my time with their shitty content either.

On Sat, Dec 17, 2016 at 6:51 PM, Sylvain BERTRAND
 wrote:
> On Sat, Dec 17, 2016 at 01:17:38PM +, Cág wrote:
>> Staven wrote:
>>
>> >or do you know a better compromise than webkit?
>>
>> Yeah, write our own web engine with blackjack and hookers, which is
>> mentioned on Project ideas page.
>
> c++ software pulls also a c++ compiler and runtime in their dependencies, 
> which
> are horrendous compared to a C compiler/miminal runtime.
>
> When you add up all of this, we are in no compromise but a total rip off and
> take over of those critical software components by very few corporations.
>
> Then I repeat myself, this is no compromise here, it's just being fuc*ed all
> the way.
>
> If I want to sabotage or take over a software component, I'll migrate it to 
> c++
> and push hard on a brain damaged object oriented design to scare away any
> reasonably skilled coders. Moreover that design will be so fuc*ed up, it will
> generate tons of bugs and security holes to justify a good salary for idiotic
> bosses (or as corrupted as their coders).
>
> The real pb are web coders: many of the web sites could have a noscript portal
> without losing significant functionalities. The web sites which _must_ have a
> rich GUI (hence a dynamic browser, for instance soundcloud) should provide an
> simple web API (without that pile of cra* which is OAuth).
>
> I'm about to sue some french administrations over their www sites and my 
> french
> bank www sites for their lack of interop with light www browsers.
>
> --
> Sylvain
>

Re: [dev] [surf] Webkit2 with proxy server

[Sylvain BERTRAND]

On Sat, Dec 17, 2016 at 01:17:38PM +, Cág wrote:
> Staven wrote:
> 
> >or do you know a better compromise than webkit?
> 
> Yeah, write our own web engine with blackjack and hookers, which is
> mentioned on Project ideas page.

c++ software pulls also a c++ compiler and runtime in their dependencies, which
are horrendous compared to a C compiler/miminal runtime.

When you add up all of this, we are in no compromise but a total rip off and
take over of those critical software components by very few corporations.

Then I repeat myself, this is no compromise here, it's just being fuc*ed all
the way.

If I want to sabotage or take over a software component, I'll migrate it to c++
and push hard on a brain damaged object oriented design to scare away any
reasonably skilled coders. Moreover that design will be so fuc*ed up, it will
generate tons of bugs and security holes to justify a good salary for idiotic
bosses (or as corrupted as their coders).

The real pb are web coders: many of the web sites could have a noscript portal
without losing significant functionalities. The web sites which _must_ have a
rich GUI (hence a dynamic browser, for instance soundcloud) should provide an
simple web API (without that pile of cra* which is OAuth).

I'm about to sue some french administrations over their www sites and my french
bank www sites for their lack of interop with light www browsers.

-- 
Sylvain

Re: [dev] [surf] Webkit2 with proxy server

[Cág]

Staven wrote:


or do you know a better compromise than webkit?


Yeah, write our own web engine with blackjack and hookers, which is 
mentioned

on Project ideas page.

Cág

Re: [dev] [surf] Webkit2 with proxy server

[Sylvain BERTRAND]

Is there anybody who dared to think that webkit-anything or geecko-anything is
in no way suckless due to their c++ implementation?

-- 
Sylvain

Re: [dev] [surf] Webkit2 with proxy server

[Ian Macdonald]

On Sat, 17 Dec 2016 09:27:02 +0800
Ivan Tham  wrote:

> On Sat, Dec 17, 2016 at 01:19:54AM +0100, Quentin Rameau wrote:
> >> Hi,  
> >Hi Ian,
> >  
> >> I have been using 'standard' surf through a proxy server for some
> >> time by just setting 'http_proxy' in the env. This doesn't work
> >> with webkit2. Does anyone know a fix? Otherwise surf2 seems to be
> >> working fine.  
> >Works for me.  
> 
> Works for me too.
> 
> >With webkit2gtk, the proxy handling is delegated down to libsoup
> >which uses libproxy which supports http_proxy.  
> 
> Yeah, libsoup automatically sets the proxy with libproxy by detecting
> the environment variables but in surf1, it's hardcoded.
> 
> But I think surf2 uses https_proxy and no_proxy too but it isn't
> documented in the manual.
> 

Thanks for that. I don't have libproxy which is likely the problem.
I've just downloaded a copy and will work on it from there.

cheers

Re: [dev] [surf] Webkit2 with proxy server

[Ivan Tham]

On Sat, Dec 17, 2016 at 01:19:54AM +0100, Quentin Rameau wrote:

Hi,

Hi Ian,


I have been using 'standard' surf through a proxy server for some time
by just setting 'http_proxy' in the env. This doesn't work with
webkit2. Does anyone know a fix? Otherwise surf2 seems to be working
fine.

Works for me.


Works for me too.


With webkit2gtk, the proxy handling is delegated down to libsoup which
uses libproxy which supports http_proxy.


Yeah, libsoup automatically sets the proxy with libproxy by detecting
the environment variables but in surf1, it's hardcoded.

But I think surf2 uses https_proxy and no_proxy too but it isn't
documented in the manual.

--
Do what you like, like what you do.  -- Pickfire

Re: [dev] [surf] Webkit2 with proxy server

[Quentin Rameau]

> Hi,
Hi Ian,

> I have been using 'standard' surf through a proxy server for some time
> by just setting 'http_proxy' in the env. This doesn't work with
> webkit2. Does anyone know a fix? Otherwise surf2 seems to be working
> fine.
Works for me.

With webkit2gtk, the proxy handling is delegated down to libsoup which
uses libproxy which supports http_proxy.

Maybe your system isn't built to reflect that (although I think libsoup
*requires* that libproxy dependency).

Please give more information if that's not the case, we can't give you
a fix to “it doesn't work”.

> Thanks
You're welcome

[dev] [surf] Webkit2 with proxy server

[Ian Macdonald]

Hi,

I have been using 'standard' surf through a proxy server for some time
by just setting 'http_proxy' in the env. This doesn't work with webkit2.
Does anyone know a fix? Otherwise surf2 seems to be working fine.

Thanks