Re: Code signing to help with Windows virus false positives
On Tuesday, 11 October 2016 at 06:09:03 UTC, Thomas Mader wrote: I worked with NSIS and InnoSetup. InnoSetup is much cleaner and easier. At work we switched from NSIS to InnoSetup and we create MSI packages from NSIS and InnoSetup packages IIRC. I think it's better to go with InnoSetup because it might be more easy and probably more powerful than building MSI directly. But I don't have any experience with building an MSI installer and the feature set of MSI. We are also signing the installer and all exe and DLLs inside. I was right. We create the MSI package out of the InnoSetup executable with a minimal xml config for WiX.
Re: Communication between 2 Socket listener on 2 different port with one program and server.
On Tuesday, 11 October 2016 at 16:59:56 UTC, vino wrote:
Hi All,
Need your help, on the below request.
Requirement:
Server:
2 socket listening to 2 different ports with one main program
Socket1 Port1 (Used for receiving user request(user data))
Socket2 Port2 (Used for system request(system data))
User request arrives via Socket1:Port1 needs to be sent
Socket2:Port2
Once the request arrives then the request has to be sent to the
Manger(another server) via Socket2:Port2
I was able to program to run multiple socket and send request
to Socket1:Port1 but not able to send the same request to
Socket2:Port2 tried both options sendTo and receiveFrom but no
luck.
Note: the user request should to directly reach the
Manger(another server) it should is always follow the data
communication layer which is Socket2:Port2 as the
server Manger will connect via Socket2:Port2(only) to receive
data.
void main () {
auto ext = new Thread(&ext).start();
auto int = new Thread(&int).start();
}
void ext () {
ushort extport = 1120;
Socket ext;
char[1024] buf;
Address mainserver = new InternetAddress("server1", 1121);
ext = new TcpSocket();
ext.bind(1120);
ext.listen(1);
ext.accpet();
ext.receive(buf[]);
writeln(buf[0..1024]);
ext.sendTo(buf[0..1024], SocketFlags.NONE, mainserver);
There's quite a few things wrong with this, I'm guessing you
don't have much experience with socket programming, but that's
ok, everyone's gotta start somewhere. You should read some
articles on socket programming, but I'll give you a few
corrections for your example.
void ext () {
ushort extport = 1120;
Address mainserver = new InternetAddress("server1", 1121);
ext = new TcpSocket();
ext.bind(1120);
ext.listen(1);
Not sure why you are using "server1" here, the listen address
acts as a "filter" on where you accept connections from. You
probably want to allow connections from any ip address in which
case you would want to pass the "any" address. You probably want
to create this socket more like this:
auto listenAddress = new
InternetAddress(InternetAddress.ADDR_ANY, 1121);
Socket listenSocket = new Socket(listenAddress.family,
SocketType.STREAM, ProtocolType.TCP);
listenSocket.bind(listenAddress);
listenSocket.listen(8); // lookup "listen" function to understand
what the "backlog" argument is
Another common address to use is the LOOPBACK address, which
means you only accept connections from the local machine (not
from any remote machine)
ext.accpet();
Here you've missed the fact that ext.accept actually returns the
socket you can call send/receive on. Here's what you should have
done:
Socket dataSocket = listenSocket.accpet();
You can't actually send/receive data on the listen socket. You
will have 1 listen socket that's listening for connections.
Every time you get a connection, the accept function will return
a new socket that you can send/receive data with for that
connection.
ext.receive(buf[]);
If you call "receive" on the data socket, you are now blocking
the listen socket from accepting more connections. That may be
ok for your application, but for some applications, they will
start a new thread to handle the data socket, and put the listen
socket accept into a loop, something like this:
while(true) {
Socket dataSocket = listenSocket.accept();
// now pass the data socket to a new thread and call receive
on that thread
// in the meantime, call accept again for any new connections
that may come in
}
// The dataSocket thread can then call receive, and print the
contents to the console like you had in your example.
void dataSocketThread()
{
ubyte[1024] buf;
auto received = dataSocket.receive(buf);
writeln(buf[0..received]);
}
// Now if you want to send this data to the other listen socket,
you'll need to create a new socket, call connect, then you can
call send
Socket newDataSocket = new Socket(...).
newDataSocket.Connect(...)
newDataSocket.send(buf[0..received]);
newDataSocket.shtudown(SD_BOTH);
newDataSocket.close();
You cannot call "sendto" on a data socket. sendto is for UDP
sockets, which you are not using in this case. For more
information, lookup a tutorial on writing a UDP echo
client/server.
Some more notes, if you don't to start a new thread every time
you accept a new connection, you can use asynchronous IO. You
can start by learning the "select" function and work your way up
to more complex apis. Each OS has their own underlying
mechanisms for async io, but there are also libraries you can use
like libev, libevent, libuv.
There's alot to learn about socket programming, this is just the
beginning. I tried to throw together a fair bit of information
in a little amount of time, hopefully you'll be able to take this
information and build on it. Good luck.
Re: Any relation?
On Tuesday, 11 October 2016 at 18:13:53 UTC, Andrei Alexandrescu wrote: http://indianautosblog.com/2016/10/most-powerful-suzuki-swift-produces-350-hp-25 -- Andrei Obviously the ECU is programmed in D. oh wait...
What influenced D? - goals, ideas, concepts, language features?
Hi list, I'm liking D as I keep using it (still new to it), and interested in how it evolved, hence this question. I have seen the Wikipedia article about D: https://en.wikipedia.org/wiki/D_(programming_language) which mentions language influences (right sidebar). I've also read some other articles on dlang sites, but thought of posting this question, to hear interesting stuff from people who know more about this. Thanks to all who reply.
Re: Any relation?
On 10/11/2016 12:53 PM, Dennis Ritchie wrote: On Tuesday, 11 October 2016 at 19:43:07 UTC, cym13 wrote: On Tuesday, 11 October 2016 at 18:58:09 UTC, Dennis Ritchie wrote: On Tuesday, 11 October 2016 at 18:13:53 UTC, Andrei Alexandrescu wrote: http://indianautosblog.com/2016/10/most-powerful-suzuki-swift-produces-350-hp-25 -- Andrei Yes, definitely. http://dlanguage-z.com/about.html Hmm... care to explain? It might sound somewhat obvious but this is written in Japanese. Here's how you can explain the connection D and this car: http://imgur.com/cG0IWOp I can also find the words "system", "main", and "garage" [sic]. ;) Ali
Re: Any relation?
On Tuesday, 11 October 2016 at 19:43:07 UTC, cym13 wrote: On Tuesday, 11 October 2016 at 18:58:09 UTC, Dennis Ritchie wrote: On Tuesday, 11 October 2016 at 18:13:53 UTC, Andrei Alexandrescu wrote: http://indianautosblog.com/2016/10/most-powerful-suzuki-swift-produces-350-hp-25 -- Andrei Yes, definitely. http://dlanguage-z.com/about.html Hmm... care to explain? It might sound somewhat obvious but this is written in Japanese. Here's how you can explain the connection D and this car: http://imgur.com/cG0IWOp
Re: Any relation?
On Tuesday, 11 October 2016 at 18:58:09 UTC, Dennis Ritchie wrote: On Tuesday, 11 October 2016 at 18:13:53 UTC, Andrei Alexandrescu wrote: http://indianautosblog.com/2016/10/most-powerful-suzuki-swift-produces-350-hp-25 -- Andrei Yes, definitely. http://dlanguage-z.com/about.html Hmm... care to explain? It might sound somewhat obvious but this is written in Japanese.
Re: Any relation?
On Tuesday, 11 October 2016 at 18:13:53 UTC, Andrei Alexandrescu wrote: http://indianautosblog.com/2016/10/most-powerful-suzuki-swift-produces-350-hp-25 -- Andrei Yes, definitely. http://dlanguage-z.com/about.html
Re: Any relation?
On Tuesday, 11 October 2016 at 18:13:53 UTC, Andrei Alexandrescu wrote: http://indianautosblog.com/2016/10/most-powerful-suzuki-swift-produces-350-hp-25 -- Andrei There is no Kenji Hara in the team, so I would say no :)
Re: Any relation?
On 10/11/2016 11:13 AM, Andrei Alexandrescu wrote: http://indianautosblog.com/2016/10/most-powerful-suzuki-swift-produces-350-hp-25 -- Andrei I think so: - D-Language: check - Most powerful: check - Japan: check - squeeze out [...] power: check - not street-legal: check (similar to D being a gun at a knife fight ;) ) And this changes the programming language car analogy for D. Now we know how to answer that question. :) Ali
Any relation?
http://indianautosblog.com/2016/10/most-powerful-suzuki-swift-produces-350-hp-25 -- Andrei
Re: Can you shrink it further?
On Tuesday, 11 October 2016 at 16:13:45 UTC, Andrei Alexandrescu wrote: On 10/11/16 11:15 AM, Stefan Koch wrote: I will now run this problem through STOKE. Let's see what it comes up with :) http://stoke.stanford.edu you mean? That would be cool. Keep us posted! -- Andrei Yep I mean that one. It will take a while to work out the right cost-functions. I'll do a PR as soon as this bears fruit.
Communication between 2 Socket listener on 2 different port with one program and server.
Hi All,
Need your help, on the below request.
Requirement:
Server:
2 socket listening to 2 different ports with one main program
Socket1 Port1 (Used for receiving user request(user data))
Socket2 Port2 (Used for system request(system data))
User request arrives via Socket1:Port1 needs to be sent
Socket2:Port2
Once the request arrives then the request has to be sent to the
Manger(another server) via Socket2:Port2
I was able to program to run multiple socket and send request to
Socket1:Port1 but not able to send the same request to
Socket2:Port2 tried both options sendTo and receiveFrom but no
luck.
Note: the user request should to directly reach the
Manger(another server) it should is always follow the data
communication layer which is Socket2:Port2 as the
server Manger will connect via Socket2:Port2(only) to receive
data.
void main () {
auto ext = new Thread(&ext).start();
auto int = new Thread(&int).start();
}
void ext () {
ushort extport = 1120;
Socket ext;
char[1024] buf;
Address mainserver = new InternetAddress("server1", 1121);
ext = new TcpSocket();
ext.bind(1120);
ext.listen(1);
ext.accpet();
ext.receive(buf[]);
writeln(buf[0..1024]);
ext.sendTo(buf[0..1024], SocketFlags.NONE, mainserver);
}
void int () {
ushort intport = 1121;
Socket int;
char[1024] buf;
Address mainserver = new InternetAddress("server1", 1120);
Address manager = new InternetAddress("server1", 1120);
int = new TcpSocket();
int.bind(1120);
int.listen(1);
int.accpet();
int.receive(buf[0..1024], SocketFlags.NONE, mainserver);
writeln(buf[0..1024]);
int.sendTo(buf[0..1024], SocketFlags.NONE, manager);
}
From,
Vino.B
Re: Can you shrink it further?
On 10/11/16 11:15 AM, Stefan Koch wrote: I will now run this problem through STOKE. Let's see what it comes up with :) http://stoke.stanford.edu you mean? That would be cool. Keep us posted! -- Andrei
Re: Can you shrink it further?
On Tuesday, 11 October 2016 at 15:08:34 UTC, Andrei Alexandrescu wrote: On 10/10/2016 11:00 PM, Stefan Koch wrote: [...] Looked at this, still seems to generate a jump forward with ldc. Also, why do you leave a fallthrough path? Progress needs to be made on all paths, otherwise we have infinite loops. I forgot that the fall-trough did no longer end in Lend; That forward jump to Lend is a very common and therefore predicted branch. I will now run this problem through STOKE. Let's see what it comes up with :)
Re: Can you shrink it further?
On Tuesday, 11 October 2016 at 15:08:34 UTC, Andrei Alexandrescu wrote: Looked at this, still seems to generate a jump forward with ldc. ldc.intrinsics.llvm_expect might help to influence basic block layout. — David
Re: Can you shrink it further?
On 10/10/2016 11:00 PM, Stefan Koch wrote:
void popFront3(ref char[] s) @trusted pure nothrow {
immutable c = s[0];
uint char_length = 1;
if (c < 127)
{
Lend :
s = s.ptr[char_length .. s.length];
} else {
if ((c & b01100_) == 0b1000_)
{
//just skip one in case this is not the beginning of a code-point
char
goto Lend;
}
if (c < 192)
{
char_length = 2;
goto Lend;
}
if (c < 240)
{
char_length = 3;
goto Lend;
}
if (c < 248)
{
char_length = 4;
goto Lend;
}
}
}
Looked at this, still seems to generate a jump forward with ldc. Also,
why do you leave a fallthrough path? Progress needs to be made on all
paths, otherwise we have infinite loops.
Andrei
Re: Can you shrink it further?
On 10/11/2016 10:49 AM, Matthias Bentrup wrote:
void popFrontAsmIntel(ref char[] s) @trusted pure nothrow {
immutable c = s[0];
if (c < 0x80) {
s = s[1 .. $];
} else {
uint l = void;
asm pure nothrow @nogc {
mov EAX, 1;
mov BL, 0xf8-1;
sub BL, c;
cmp BL, 0xf8-0xc0;
adc EAX, 0;
cmp BL, 0xf8-0xe0;
adc EAX, 0;
cmp BL, 0xf8-0xf0;
adc EAX, 0;
mov l, EAX;
}
s = s[l <= $ ? l : $ .. $];
}
}
Did you take a look at the codegen on http://ldc.acomirei.ru? It's huge.
-- Andrei
Re: Can you shrink it further?
On Tuesday, 11 October 2016 at 14:49:28 UTC, Matthias Bentrup
wrote:
This is the result I'd like to get, but I can't find a way to
write it without inline assembly :(
void popFrontAsmIntel(ref char[] s) @trusted pure nothrow {
immutable c = s[0];
if (c < 0x80) {
s = s[1 .. $];
} else {
uint l = void;
asm pure nothrow @nogc {
mov EAX, 1;
mov BL, 0xf8-1;
sub BL, c;
cmp BL, 0xf8-0xc0;
adc EAX, 0;
cmp BL, 0xf8-0xe0;
adc EAX, 0;
cmp BL, 0xf8-0xf0;
adc EAX, 0;
mov l, EAX;
}
s = s[l <= $ ? l : $ .. $];
}
}
This takes 180us.
Baseline takes 124us.
Re: Can you shrink it further?
On 10/11/2016 05:45 AM, Temtaime wrote:
void popFront7(ref char[] s) @trusted pure nothrow
{
import core.bitop;
auto v = 7 - bsr(~s[0] | 1);
s = s[v > 6 ? 1 : (v ? (v > s.length ? s.length : v) : 1)..$];
}
Please check this.
Thanks. This does a lot of work on the frequent path c < 0x80:
pure nothrow @trusted void example.popFront7(ref char[]):
movq8(%rdi), %rax
movzbl (%rax), %ecx
xorq$254, %rcx
orq $1, %rcx
bsrq%rcx, %rcx
notl%ecx
addl$8, %ecx
cmpl$6, %ecx
jg .LBB0_2
testl %ecx, %ecx
je .LBB0_2
movslq %ecx, %rdx
movq(%rdi), %rcx
cmpq%rcx, %rdx
cmovaq %rcx, %rdx
jmp .LBB0_3
.LBB0_2:
movq(%rdi), %rcx
movl$1, %edx
.LBB0_3:
addq%rdx, %rax
subq%rdx, %rcx
movq%rcx, (%rdi)
movq%rax, 8(%rdi)
retq
So I changed it to:
void popFront7(ref char[] s) @trusted pure nothrow
{
immutable c = s[0];
if (c < 0x80) {
s = s.ptr[1 .. s.length];
} else {
import core.bitop;
uint v = 7 - bsr(~c | (c > 0xfd) << 6u);
s = s.ptr[v > s.length ? s.length : v .. s.length];
}
}
That's about as large as the baseline.
Andrei
Re: Can you shrink it further?
On Tuesday, 11 October 2016 at 14:24:56 UTC, Andrei Alexandrescu
wrote:
On 10/11/2016 03:30 AM, Matthias Bentrup wrote:
A branch-free version:
void popFront4(ref char[] s) @trusted pure nothrow {
immutable c = s[0];
uint char_length = 1 + (c >= 192) + (c >= 240) + (c >= 248);
s = s.ptr[char_length .. s.length];
}
Theoretically the char_length could be computed with three sub
and addc
instructions, but no compiler is smart enough to detect that.
Interesting. 0x80 should be special-cased and you forgot to
check the bounds. So:
void popFront4(ref char[] s) @trusted pure nothrow {
immutable c = s[0];
if (c < 0x80) {
s = s.ptr[1 .. s.length];
} else {
uint l = 1 + (c >= 192) + (c >= 240) + (c >= 248);
s = s.ptr[l <= s.length ? l : s.length .. s.length];
}
}
This generated 27 instructions, i.e. same as the baseline. See:
http://ldc.acomirei.ru/#compilers:!((compiler:ldc,options:'-release+-O3+-boundscheck%3Doff',sourcez:G4ewlgJgBADiMDEBOIB2AXAjACiQUwDMoBjACwEMkBtAXSgGcBKKAAXSQFd709oYP8UVCHSkUAdygBvAFBQoYALaKO6cgCMANnhJQAvAyoAGGgG45CotmJQAPFCMAPABxHms%2BfPr6GAOhjsVJhQvr5%2B2qgA5qJmFgC%2BUHia9DoenkpwSOgkIPi%2B6mDo8OaeUBxgGAo%2BAOwcUAC0UOr0SNgAfjYAPlCYHIwl6YqZ2dwQvuSakbmFpIoD8mBWYFAAfFAAbH1VBpjzDD70/oGKFdhgADTheFGizKFXN6Sx8nEyrzKgkLDwyGjoAEy4QgkCjUOjcJDMNicbi8WACHTCUQSaQWJQqNRaHQ2AwQ4zPSxQax2BwuNwWNLyAD0VICSAU3i4cKKUHIn2gHFQXLwxDw9HolAAnk0QJyIN4yDyANYVSK%2BCxedgHdhHajBe4Q3wRaJPAaveRJFKE4l6AxOAgERgUhVQGlNFplVAQQgVOEEXIOG0Q5VIVVBEJhTXamJ6iyGvDW0oZXLZYi5PD5QrwKAALntSD25FUICginozRqDXT7WI/RtiyJ2DzBfs/2Y3Sr%2Be8a3WjCtpUpnhpAElUMAJl8AKoAFQQ9WcNvk1e8Oz2%2BsGwwY6DGEymSBmcy9StxKrpVBOqEbzUuQeuOrugZVwd18TeMiAA)),filterAsm:(commentOnly:!t,directives:!t,labels:!t),version:3
But it doesn't seem to check for all errors.
Andrei
This is the result I'd like to get, but I can't find a way to
write it without inline assembly :(
void popFrontAsmIntel(ref char[] s) @trusted pure nothrow {
immutable c = s[0];
if (c < 0x80) {
s = s[1 .. $];
} else {
uint l = void;
asm pure nothrow @nogc {
mov EAX, 1;
mov BL, 0xf8-1;
sub BL, c;
cmp BL, 0xf8-0xc0;
adc EAX, 0;
cmp BL, 0xf8-0xe0;
adc EAX, 0;
cmp BL, 0xf8-0xf0;
adc EAX, 0;
mov l, EAX;
}
s = s[l <= $ ? l : $ .. $];
}
}
Re: Can you shrink it further?
On Tuesday, 11 October 2016 at 14:24:56 UTC, Andrei Alexandrescu
wrote:
On 10/11/2016 03:30 AM, Matthias Bentrup wrote:
A branch-free version:
void popFront4(ref char[] s) @trusted pure nothrow {
immutable c = s[0];
uint char_length = 1 + (c >= 192) + (c >= 240) + (c >= 248);
s = s.ptr[char_length .. s.length];
}
void popFront4(ref char[] s) @trusted pure nothrow {
immutable c = s[0];
if (c < 0x80) {
s = s.ptr[1 .. s.length];
} else {
uint l = 1 + (c >= 192) + (c >= 240) + (c >= 248);
s = s.ptr[l <= s.length ? l : s.length .. s.length];
}
}
This generated 27 instructions, i.e. same as the baseline. See:
http://ldc.acomirei.ru/#compilers:!((compiler:ldc,options:'-release+-O3+-boundscheck%3Doff',sourcez:G4ewlgJgBADiMDEBOIB2AXAjACiQUwDMoBjACwEMkBtAXSgGcBKKAAXSQFd709oYP8UVCHSkUAdygBvAFBQoYALaKO6cgCMANnhJQAvAyoAGGgG45CotmJQAPFCMAPABxHms%2BfPr6GAOhjsVJhQvr5%2B2qgA5qJmFgC%2BUHia9DoenkpwSOgkIPi%2B6mDo8OaeUBxgGAo%2BAOwcUAC0UOr0SNgAfjYAPlCYHIwl6YqZ2dwQvuSakbmFpIoD8mBWYFAAfFAAbH1VBpjzDD70/oGKFdhgADTheFGizKFXN6Sx8nEyrzKgkLDwyGjoAEy4QgkCjUOjcJDMNicbi8WACHTCUQSaQWJQqNRaHQ2AwQ4zPSxQax2BwuNwWNLyAD0VICSAU3i4cKKUHIn2gHFQXLwxDw9HolAAnk0QJyIN4yDyANYVSK%2BCxedgHdhHajBe4Q3wRaJPAaveRJFKE4l6AxOAgERgUhVQGlNFplVAQQgVOEEXIOG0Q5VIVVBEJhTXamJ6iyGvDW0oZXLZYi5PD5QrwKAALntSD25FUICginozRqDXT7WI/RtiyJ2DzBfs/2Y3Sr%2Be8a3WjCtpUpnhpAElUMAJl8AKoAFQQ9WcNvk1e8Oz2%2BsGwwY6DGEymSBmcy9StxKrpVBOqEbzUuQeuOrugZVwd18TeMiAA)),filterAsm:(commentOnly:!t,directives:!t,labels:!t),version:3
But it doesn't seem to check for all errors.
Andrei
It's much slower.
Because of the flag-storing instructions.
Re: Can you shrink it further?
On 10/11/2016 03:30 AM, Matthias Bentrup wrote:
A branch-free version:
void popFront4(ref char[] s) @trusted pure nothrow {
immutable c = s[0];
uint char_length = 1 + (c >= 192) + (c >= 240) + (c >= 248);
s = s.ptr[char_length .. s.length];
}
Theoretically the char_length could be computed with three sub and addc
instructions, but no compiler is smart enough to detect that.
Interesting. 0x80 should be special-cased and you forgot to check the
bounds. So:
void popFront4(ref char[] s) @trusted pure nothrow {
immutable c = s[0];
if (c < 0x80) {
s = s.ptr[1 .. s.length];
} else {
uint l = 1 + (c >= 192) + (c >= 240) + (c >= 248);
s = s.ptr[l <= s.length ? l : s.length .. s.length];
}
}
This generated 27 instructions, i.e. same as the baseline. See:
http://ldc.acomirei.ru/#compilers:!((compiler:ldc,options:'-release+-O3+-boundscheck%3Doff',sourcez:G4ewlgJgBADiMDEBOIB2AXAjACiQUwDMoBjACwEMkBtAXSgGcBKKAAXSQFd709oYP8UVCHSkUAdygBvAFBQoYALaKO6cgCMANnhJQAvAyoAGGgG45CotmJQAPFCMAPABxHms%2BfPr6GAOhjsVJhQvr5%2B2qgA5qJmFgC%2BUHia9DoenkpwSOgkIPi%2B6mDo8OaeUBxgGAo%2BAOwcUAC0UOr0SNgAfjYAPlCYHIwl6YqZ2dwQvuSakbmFpIoD8mBWYFAAfFAAbH1VBpjzDD70/oGKFdhgADTheFGizKFXN6Sx8nEyrzKgkLDwyGjoAEy4QgkCjUOjcJDMNicbi8WACHTCUQSaQWJQqNRaHQ2AwQ4zPSxQax2BwuNwWNLyAD0VICSAU3i4cKKUHIn2gHFQXLwxDw9HolAAnk0QJyIN4yDyANYVSK%2BCxedgHdhHajBe4Q3wRaJPAaveRJFKE4l6AxOAgERgUhVQGlNFplVAQQgVOEEXIOG0Q5VIVVBEJhTXamJ6iyGvDW0oZXLZYi5PD5QrwKAALntSD25FUICginozRqDXT7WI/RtiyJ2DzBfs/2Y3Sr%2Be8a3WjCtpUpnhpAElUMAJl8AKoAFQQ9WcNvk1e8Oz2%2BsGwwY6DGEymSBmcy9StxKrpVBOqEbzUuQeuOrugZVwd18TeMiAA)),filterAsm:(commentOnly:!t,directives:!t,labels:!t),version:3
But it doesn't seem to check for all errors.
Andrei
Re: Can you shrink it further?
On Tuesday, 11 October 2016 at 14:16:54 UTC, Andrei Alexandrescu wrote: On 10/11/2016 04:57 AM, Stefan Koch wrote: Yours runs with 790 us best time. bsr is a real timetaker :) What inputs did you test it on? https://github.com/minimaxir/big-list-of-naughty-strings/blob/master/blns.txt Here's what I think would be a good set of requirements: * The ASCII case should be short and fast: a comparison and a branch, followed by return. This would improve a very common case and address the main issue with autodecoding. Already done * For the multibyte case, the main requirement is the code must be small. This is because it gets inlined all over the place and seldom used. * For the multibyte case, the fewer bytes in the encoding the less work. This is because more frequent multi-byte characters have generally lower codes. That is why I had the branches, generally only the first one is taken Currently front() - the other time spender in autodecoding - issues a function call on the multibyte case. That makes the code of front() itself small, at the cost of more expensive multibyte handling. I think at some point we have to cache the length of the last decoded char, Otherwise we are throwing work away. However that will only work within a RangeWrapper-Struct
[your code here]
judr jpg
Re: Can you shrink it further?
On 10/11/2016 04:57 AM, Stefan Koch wrote: Yours runs with 790 us best time. bsr is a real timetaker :) What inputs did you test it on? Here's what I think would be a good set of requirements: * The ASCII case should be short and fast: a comparison and a branch, followed by return. This would improve a very common case and address the main issue with autodecoding. * For the multibyte case, the main requirement is the code must be small. This is because it gets inlined all over the place and seldom used. * For the multibyte case, the fewer bytes in the encoding the less work. This is because more frequent multi-byte characters have generally lower codes. Currently front() - the other time spender in autodecoding - issues a function call on the multibyte case. That makes the code of front() itself small, at the cost of more expensive multibyte handling. Andrei
Re: Can you shrink it further?
On Tuesday, 11 October 2016 at 10:01:41 UTC, Stefan Koch wrote:
On Tuesday, 11 October 2016 at 09:45:11 UTC, Temtaime wrote:
Sorry this was also a type in the code.
void popFront7(ref char[] s) @trusted pure nothrow
{
import core.bitop;
auto v = 7 - bsr(~s[0] | 1);
s = s[v > 6 ? 1 : (v ? (v > s.length ? s.length : v) :
1)..$];
}
Please check this.
162 us
The branching, it hurts my eyes!
Something like the following should give correct (assuming I
haven't written bad logic) branchless results with
architecture-optimised max calls. Note that the minus/plus 1
operation on the third line will ensure with the sign
multiplication that values of 7 will map to 1, whereas for all
other values it's an extra operation. But the advantage is that
you're not sticking three branches in close proximity to each
other, so you will never get a branch predictor fail. (Of note,
any performance test for these functions should test with data
designed to fail the branching code I quoted, keeping in mind
that desktop Intel processors have a four-state branch predictor.
I've not performance tested it myself, but this will certainly
run faster on the AMD Jaguar processors than a version with
branching checks.)
int v = 7 - bsr( ~s[0] | 1 );
int sign = ( (v - 7) >> 31 );
v = ( v - 1 ) * sign + 1;
str = str[ min( v, s.length ) .. $ ];
Re: color lib
On Tuesday, 11 October 2016 at 12:14:37 UTC, Manu wrote: Oh dear... thanks for digging that up. I didn't know the web had a standard for alpha. Certainly 0xAARRGGBB has been used in windows code for as long as I've been programming... but now there's a competing #RRGGBBAA version... How to resolve this? I guess, go with the web? I should probably change it to the CSS4 way. My idea is still to use a template: colorFromString!"rgba" or colorFromString!"argb" (please notice that AFAIK, the second most used way is actually "abgr" rather than "argb" - because of byte-order) And I think it's a good idea to set template argument to some default. MS is not sure about this, anyway. Read carefully this: https://msdn.microsoft.com/en-us/library/windows/desktop/ms534427(v=vs.85).aspx VS: https://msdn.microsoft.com/en-us/library/ee505694(v=winembedded.60).aspx VS: https://msdn.microsoft.com/it-it/library/windows/desktop/dd183449(v=vs.85).aspx OpenGL use rgba (and abgr?) directX argb: https://www.opengl.org/wiki/Image_Format https://www.opengl.org/wiki/Direct3D_Compatibility Andrea
Re: color lib
On 11 October 2016 at 18:10, Andrea Fontana via Digitalmars-d wrote: > On Monday, 10 October 2016 at 23:26:53 UTC, Manu wrote: >> >> I'm not sure why it matters what format the colour you have is... >> Strings are in the form #RRGGBB, or #AARRGGBB. That is all. >> It's the standard I've seen used everywhere ever, including the web, >> which is a pretty good precedent :P > > > If the web is a good precedent (CSS4 specs): > "The first 6 digits are interpreted identically to the 6-digit notation. The > last pair of digits, interpreted as a hexadecimal number, specifies the > alpha channel of the color, where 00 represents a fully transparent color > and ff represent a fully opaque color." > > https://drafts.csswg.org/css-color/#hex-notation > > CSS3 doesn't support hex string with alpha but they suggest you to use > rgba() function. I think argb() doesn't exists instead. > https://www.w3.org/TR/2011/REC-css3-color-20110607/#rgba-color > > Chrome 52 supports it: > https://googlechrome.github.io/samples/css-alpha-channel/ > > Android instead: > https://developer.android.com/reference/android/graphics/Color.html#parseColor(java.lang.String) > "Parse the color string, and return the corresponding color-int. If the > string cannot be parsed, throws an IllegalArgumentException exception. > Supported formats are: #RRGGBB #AARRGGBB [...]" > > Please notice that on PNG file format rgba is quite common (also on bmp with > semi-official apha support) > PNG: http://www.libpng.org/pub/png/book/chapter08.html Oh dear... thanks for digging that up. I didn't know the web had a standard for alpha. Certainly 0xAARRGGBB has been used in windows code for as long as I've been programming... but now there's a competing #RRGGBBAA version... How to resolve this? I guess, go with the web? I should probably change it to the CSS4 way.
Re: Can you shrink it further?
On Tuesday, 11 October 2016 at 09:45:11 UTC, Temtaime wrote:
Sorry this was also a type in the code.
void popFront7(ref char[] s) @trusted pure nothrow
{
import core.bitop;
auto v = 7 - bsr(~s[0] | 1);
s = s[v > 6 ? 1 : (v ? (v > s.length ? s.length : v) : 1)..$];
}
Please check this.
162 us
Re: Can you shrink it further?
On Tuesday, 11 October 2016 at 09:13:10 UTC, Stefan Koch wrote:
On Tuesday, 11 October 2016 at 08:57:46 UTC, Stefan Koch wrote:
On Tuesday, 11 October 2016 at 08:44:04 UTC, Temtaime wrote:
void popFront1(ref char[] s) @trusted pure nothrow
{
import core.bitop, std.algorithm;
auto v = bsr(~s[0] | 1);
s = s[clamp(v, 1, v > 6 ? 1 : $)..$];
}
Seems to be less if i'm not wrong.
Yours runs with 790 us best time.
bsr is a real timetaker :)
CORRECTION this is not bsr's fault.
It's most likely clamp.
I am compiling with dmd and dmd is not as good in optimizing
when templates are in the mix.
Sorry this was also a type in the code.
void popFront7(ref char[] s) @trusted pure nothrow
{
import core.bitop;
auto v = 7 - bsr(~s[0] | 1);
s = s[v > 6 ? 1 : (v ? (v > s.length ? s.length : v) : 1)..$];
}
Please check this.
Re: Can you shrink it further?
On Tuesday, 11 October 2016 at 08:57:46 UTC, Stefan Koch wrote:
On Tuesday, 11 October 2016 at 08:44:04 UTC, Temtaime wrote:
void popFront1(ref char[] s) @trusted pure nothrow
{
import core.bitop, std.algorithm;
auto v = bsr(~s[0] | 1);
s = s[clamp(v, 1, v > 6 ? 1 : $)..$];
}
Seems to be less if i'm not wrong.
Yours runs with 790 us best time.
bsr is a real timetaker :)
CORRECTION this is not bsr's fault.
It's most likely clamp.
I am compiling with dmd and dmd is not as good in optimizing when
templates are in the mix.
Re: Can you shrink it further?
On Tuesday, 11 October 2016 at 08:44:04 UTC, Temtaime wrote:
void popFront1(ref char[] s) @trusted pure nothrow
{
import core.bitop, std.algorithm;
auto v = bsr(~s[0] | 1);
s = s[clamp(v, 1, v > 6 ? 1 : $)..$];
}
Seems to be less if i'm not wrong.
Yours runs with 790 us best time.
bsr is a real timetaker :)
Re: Can you shrink it further?
On Tuesday, 11 October 2016 at 08:17:52 UTC, Stefan Koch wrote:
Also the code produces conditional set instructions which have
a higher latency.
And worse throughput.
On my arguably a bit dated laptop:
popFront3 performs 109 us best time
and popFront4 performs with 265 us best time
Testcode :
void main()
{
import std.datetime : StopWatch;
import std.stdio;
foreach(_;0 .. 255) {
char[] test1 = (import("blns.txt")).dup;
StopWatch sw;
sw.start;
while(test1.length) popFront(test1);
sw.stop;
writeln("pf1 took ", sw.peek.usecs, "us");
sw.reset();
}
}
blns.txt is taken from
https://github.com/minimaxir/big-list-of-naughty-strings/blob/master/blns.txt
Re: Can you shrink it further?
On Tuesday, 11 October 2016 at 08:17:52 UTC, Stefan Koch wrote:
On Tuesday, 11 October 2016 at 08:03:40 UTC, Stefan Koch wrote:
On Tuesday, 11 October 2016 at 07:30:26 UTC, Matthias Bentrup
wrote:
A branch-free version:
void popFront4(ref char[] s) @trusted pure nothrow {
immutable c = s[0];
uint char_length = 1 + (c >= 192) + (c >= 240) + (c >= 248);
s = s.ptr[char_length .. s.length];
}
Theoretically the char_length could be computed with three
sub and addc instructions, but no compiler is smart enough to
detect that.
You still need to special case c < 128
as well as the follow chars.
also smaller c's are more common the bigger ones making the
branching version faster on average.
Also the code produces conditional set instructions which have
a higher latency.
And worse throughput.
void popFront1(ref char[] s) @trusted pure nothrow
{
import core.bitop, std.algorithm;
auto v = bsr(~s[0] | 1);
s = s[clamp(v, 1, v > 6 ? 1 : $)..$];
}
Seems to be less if i'm not wrong.
Re: Can you shrink it further?
On Tuesday, 11 October 2016 at 08:03:40 UTC, Stefan Koch wrote:
On Tuesday, 11 October 2016 at 07:30:26 UTC, Matthias Bentrup
wrote:
A branch-free version:
void popFront4(ref char[] s) @trusted pure nothrow {
immutable c = s[0];
uint char_length = 1 + (c >= 192) + (c >= 240) + (c >= 248);
s = s.ptr[char_length .. s.length];
}
Theoretically the char_length could be computed with three sub
and addc instructions, but no compiler is smart enough to
detect that.
You still need to special case c < 128
as well as the follow chars.
also smaller c's are more common the bigger ones making the
branching version faster on average.
Also the code produces conditional set instructions which have a
higher latency.
And worse throughput.
Re: color lib
On Monday, 10 October 2016 at 23:26:53 UTC, Manu wrote: I'm not sure why it matters what format the colour you have is... Strings are in the form #RRGGBB, or #AARRGGBB. That is all. It's the standard I've seen used everywhere ever, including the web, which is a pretty good precedent :P If the web is a good precedent (CSS4 specs): "The first 6 digits are interpreted identically to the 6-digit notation. The last pair of digits, interpreted as a hexadecimal number, specifies the alpha channel of the color, where 00 represents a fully transparent color and ff represent a fully opaque color." https://drafts.csswg.org/css-color/#hex-notation CSS3 doesn't support hex string with alpha but they suggest you to use rgba() function. I think argb() doesn't exists instead. https://www.w3.org/TR/2011/REC-css3-color-20110607/#rgba-color Chrome 52 supports it: https://googlechrome.github.io/samples/css-alpha-channel/ Android instead: https://developer.android.com/reference/android/graphics/Color.html#parseColor(java.lang.String) "Parse the color string, and return the corresponding color-int. If the string cannot be parsed, throws an IllegalArgumentException exception. Supported formats are: #RRGGBB #AARRGGBB [...]" Please notice that on PNG file format rgba is quite common (also on bmp with semi-official apha support) PNG: http://www.libpng.org/pub/png/book/chapter08.html
Re: Can you shrink it further?
On Tuesday, 11 October 2016 at 07:30:26 UTC, Matthias Bentrup
wrote:
A branch-free version:
void popFront4(ref char[] s) @trusted pure nothrow {
immutable c = s[0];
uint char_length = 1 + (c >= 192) + (c >= 240) + (c >= 248);
s = s.ptr[char_length .. s.length];
}
Theoretically the char_length could be computed with three sub
and addc instructions, but no compiler is smart enough to
detect that.
You still need to special case c < 128
as well as the follow chars.
also smaller c's are more common the bigger ones making the
branching version faster on average.
Re: Can you shrink it further?
On Tuesday, 11 October 2016 at 04:05:47 UTC, Stefan Koch wrote:
On Tuesday, 11 October 2016 at 03:58:59 UTC, Andrei
Alexandrescu wrote:
On 10/10/16 11:00 PM, Stefan Koch wrote:
On Tuesday, 11 October 2016 at 02:48:22 UTC, Andrei
Alexandrescu wrote:
[...]
If you want to skip a byte it's easy to do as well.
void popFront3(ref char[] s) @trusted pure nothrow {
immutable c = s[0];
uint char_length = 1;
if (c < 127)
{
Lend :
s = s.ptr[char_length .. s.length];
} else {
if ((c & b01100_) == 0b1000_)
{
//just skip one in case this is not the beginning of a
code-point
char
goto Lend;
}
if (c < 192)
{
char_length = 2;
goto Lend;
}
if (c < 240)
{
char_length = 3;
goto Lend;
}
if (c < 248)
{
char_length = 4;
goto Lend;
}
}
}
Affirmative. That's identical to the code in "[ ... ]" :o).
Generated code still does a jmp forward though. -- Andrei
It was not identical.
((c & b01100_) == 0b1000_))
Can be true in all of the 3 following cases.
If we do not do a jmp to return here, we cannot guarantee that
we will not skip over the next valid char.
Thereby corrupting already corrupt strings even more.
For best performance we need to leave the gotos in there.
A branch-free version:
void popFront4(ref char[] s) @trusted pure nothrow {
immutable c = s[0];
uint char_length = 1 + (c >= 192) + (c >= 240) + (c >= 248);
s = s.ptr[char_length .. s.length];
}
Theoretically the char_length could be computed with three sub
and addc instructions, but no compiler is smart enough to detect
that.
