[go-nuts] ANN: Basic test impact analysis (benchmark shows avg 29% reduction in test execution time atm)

['Markus Zimmermann' via golang-nuts]

[image: with-go-test-with-test-runner.png]

We implemented a basic test impact analysis that identifies and then 
executes affected tests for a change, e.g. for a range of commits. Our 
benchmark for the repositories we looked at shows an average 29% reduction 
in test execution time. I am very much looking forward to hearing how your 
repositories perform!

Details of the benchmark and how the analysis/command works can be found 
here: https://symflower.com/en/company/blog/2024/test-impact-analysis/. The 
approach right now is to query a diff using Git and then check which Go 
packages are affected. In the next iterations, we will bring this analysis 
down to individual function and test-case level. The eventual goal is to 
use our symbolic execution engine to allow for even deeper granularity.

At one point i like to have it as a drop-in command for `go` but right now 
execution looks like this (for all changes of the last commit to now):

```
symflower test-runner --commit-from HEAD~ -- go test -v
```

Looking forward to your feedback!

Cheers,
Markus

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0d4cf273-ad2f-4b51-ba9d-d18081535700n%40googlegroups.com.

[go-nuts] Re: Improving on unit test styles

['Markus Zimmermann' via golang-nuts]

First change for github.com/stretchr/testify has been pushed as a 
PR https://github.com/stretchr/testify/pull/1161: "Include locations of 
subtests in test failure stack traces". Please give your emoji-support if 
you agree that this is a good behavior change.

More open source changes are coming. Takes some time because we need to 
alternate between product work.
On Friday, 25 February 2022 at 19:48:19 UTC+1 Markus Zimmermann wrote:

> Hi Gophers!
>
> We were unhappy with the common unit test styles and we think we found a 
> style that has clear advantages. An in-depth comparison can be found here 
> https://symflower.com/en/company/blog/2022/better-table-driven-testing.
>
> We also added support for maintaining such tests in our VS Code extension 
> https://marketplace.visualstudio.com/items?itemName=symflower.symflower. 
> You can either use the command or the context menu item to maintain tests. 
> Here is a short introduction video 
> https://www.youtube.com/watch?v=mgWLY9DDyDE&ab_channel=Symflower
>
> There are some changes necessary to have better stack traces for 
> github.com/stretchr/testify because "t.Run" calls the test function from 
> another location. We are in the process of upstreaming them. Until then you 
> can find them in our fork at https://github.com/symflower/testify.
>
> Would appreciate your feedback on the style and extension. Would be also 
> interesting to hear other approaches and conventions that could help others 
> to write better tests.
>
> Cheers,
> Markus
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/dadfaff4-fb49-4953-80df-075f923c9032n%40googlegroups.com.

[go-nuts] Improving on unit test styles

['Markus Zimmermann' via golang-nuts]

Hi Gophers!

We were unhappy with the common unit test styles and we think we found a 
style that has clear advantages. An in-depth comparison can be found here 
https://symflower.com/en/company/blog/2022/better-table-driven-testing.

We also added support for maintaining such tests in our VS Code extension 
https://marketplace.visualstudio.com/items?itemName=symflower.symflower. 
You can either use the command or the context menu item to maintain tests. 
Here is a short introduction video 
https://www.youtube.com/watch?v=mgWLY9DDyDE&ab_channel=Symflower

There are some changes necessary to have better stack traces for 
github.com/stretchr/testify because "t.Run" calls the test function from 
another location. We are in the process of upstreaming them. Until then you 
can find them in our fork at https://github.com/symflower/testify.

Would appreciate your feedback on the style and extension. Would be also 
interesting to hear other approaches and conventions that could help others 
to write better tests.

Cheers,
Markus

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/4c52a6b6-6762-442c-b9b8-82b8f50a732dn%40googlegroups.com.

Re: [go-nuts] Re: What are the options to fake SQL database for testing?

[Markus Zimmermann]

We switched from SQLite to PostgreSQL as our "in memory" database. We use 
an SQL database so we do not have to mock DB calls and maintain an 
interface. If that is something that is interesting i could trigger a blog 
post about what we did and why?

On Tuesday, July 27, 2021 at 10:21:23 AM UTC+2 mlevi...@gmail.com wrote:

> Hi,
>
> IMO, specific mocks like DATA-DOG's tend to be complicated to use and have 
> behaviors that should not appear in a mock, which would ideally tend to 
> have no logic at all.
> There are several mock packages that you can find here 
>  [and BTW if you have time 
> to give feedback I'm currently working on my own 
> , anyway that is completely unrelated].
>
> It really depends on what you are looking for in terms of features. Do you 
> need something that is able to error on syntax problems? That keeps track 
> of the execution flow (i.e. if you insert data in your fake database, are 
> you expecting it to be able to return it to you? Under what conditions?)?
>
> Mocking complex systems like SQL databases is quite hard... What I have 
> seen many times is just to have a "seed-based" local database that is used 
> in local integration tests.
> Another, quite efficient solution is to have a test suite's database 
> initialized once and used for all subsequent unit tests. Once you have good 
> unit tests for your database-interacting low-level functions, and 
> integration tests for the whole flow, you can just mock the db behavior in 
> other higher level unit tests because you know it'll work in production.
>
> Hope this helps!
>
> Le mar. 27 juil. 2021 à 10:04, Amit Saha  a écrit :
>
>> Hi all,
>>
>> I am just looking at options to write tests for parts of my application 
>> that interacts with a SQL database. So far it seems like the community has 
>> few distinct schools of thought:
>>
>> - Mocks (For e.g. using https://pkg.go.dev/github.com/DATA-DOG/go-sqlmock
>> )
>> - In-memory "real" DB solutions - such as tidb-lite for MySQL (
>> https://github.com/WangXiangUSTC/tidb-lite)
>> - Container based functional/integration style testing
>>
>> It will be great to hear if anybody has some other experiences to share.
>>
>> Thanks,
>> Amit.
>>
>>
>> On Saturday, October 10, 2015 at 5:30:20 AM UTC+11 kyle.a...@gmail.com 
>> wrote:
>>
>>>
>>> On Thursday, October 8, 2015 at 3:54:08 PM UTC-4, vkoch...@gmail.com 
>>> wrote:
>>>
 P.S. I don’t see  embedded versions of PostgreSQL or MySql. The SQLite 
 is not quite reach SQL implementation from enterprise point of view.

>>>
>>> Maybe have a look at tidb ? (
>>> https://github.com/pingcap/tidb/blob/master/docs/USAGE.md) 
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/4fbc3ae9-b0ff-4750-bfba-1d58a1dba986n%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/5829cbca-703d-42c4-9338-5f739375568an%40googlegroups.com.

[go-nuts] [ANN] go-mutesting v1.1 - Mutation testing for Go source code

[Markus Zimmermann]

Hi gophers!

I would like to announce v1.1 of go-mutesting 
https://github.com/zimmski/go-mutesting/releases/tag/v1.1 

Changes from 1.0 to 1.1

- Mutator for the comparison operators “<”, “<=”, “>” and “>=” to catch 
off-by-one problems

- Complete blacklist of reserved and built-in keywords that should not be 
used as identifiers (reduces invalid mutations)

- Metric for how many duplicated mutations were generated

- Switched to go/loader for type checking entire packages (reduces invalid 
mutations)

- Ignore mutations that cannot be compiled (reduces invalid mutations)

- Skipped mutations now count towards the total mutation counter and not 
the duplicated counter

- Support Go 1.10 and 1.11 and drop support for Go < 1.10

- Support MacOSX

As always, I appreciate any suggestions and comments. Also, if you are 
looking for a project to improve your Go skills, you can find some really 
nice additions in the project’s tracker 
https://github.com/zimmski/go-mutesting/issues I am happy to help if you 
need any guidance.

Cheers,

Markus

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Why so many opt-out changes to test runs in 1.10?

[Markus Zimmermann]

I fully agree with Aaron. Both are fine additions but they should be opt-in 
instead of opt-out. "We" (and myself) have Vet in the linting stage of all 
of our projects. We do not want to run a static analysis in the test stage. 
Also, until the dependency issue is fixed for all possible scenarios I do 
not see this enabled either. Wasting CPU time is OK-ish, but wasting 
developer time for hunting down caching-problems is definitely not good.

On Friday, December 8, 2017 at 6:28:47 PM UTC+1, Aaron Lefkowitz wrote:
>
> There's two features that are in Go 1.10 that are opt-out new defaults. I 
> sincerely apologize for not being active enough to spot them before now.
>
> > The go test command now caches test results: if the test executable and 
> command line match a previous run and the files and environment variables 
> consulted by that run have not changed either, go test will print the 
> previous test output, replacing the elapsed time with the string 
> “(cached).” Test caching applies only to successful test results; only to 
> go test commands with an explicit list of packages; and only to command 
> lines using a subset of the -cpu, -list, -parallel, -run, -short, and -v test 
> flags. The idiomatic way to bypass test caching is to use -count=1.
>
> There was some discussion on the Github issue that performance is the key 
> concern (as it usually is when caching comes into play): 
> https://github.com/golang/go/issues/11193
> However on a follow-up discussion it appears that it's actually fairly 
> impossible to be able to nail down the dependencies properly: 
> https://github.com/golang/go/issues/22593
> So the question would be why is this being introduced as an opt-out 
> feature, where it will cause many problems until people realize that their 
> test runs are being cached and disable it, which will lead to a "remember 
> to turn this off, it causes problems" feature, which would mean the default 
> is wrong. Why not do the easy fix here? There are people with problems in 
> this area because their tests don't need to be re-run and it costs them 
> significant amounts of time, why can't this simply be a -cache flag, not 
> only is this much more intuitive than -count=1 to disable caching in 
> general, but it preserves the behavior that: In most circumstances you will 
> probably just want your tests to run, even if they were successful the last 
> time, because something has changed; env, testdata, databases, external 
> golden files, etc. Or maybe you are trying to debug a flaky test (up-enter 
> enough times to get the intermittent error back now requires a -count=1).
>
> Let me be clear: This feature is amazing, and I want it. But I do think 
> the default is wrong.
>
> > The go test command now automatically runs go vet on the package being 
> tested, to identify significant problems before running the test. Any such 
> problems are treated like build errors and prevent execution of the test. 
> Only a high-confidence subset of the available go vet checks are enabled 
> for this automatic check. To disable the running of go vet, use go test 
> -vet=off.
>
> This is another opt-out change, why shouldn't it be an opt-in change? 
> Tools can easily be built around this to ensure that when a particular 
> project runs its tests it runs go vet first, this is trivial. You could 
> argue that the way this was implemented (being close to the toolchain) 
> makes it favorable to that solution because it's being run when there's 
> cycles to spare (during link) and therefore the performance impact should 
> be negligible. But then why just go vet? Why a subset of its checks? Why 
> not golint? If it's going to prevent me from running my tests and I have to 
> disable it in order to run these tests (despite being a valid Go program), 
> why are they not compiler errors?
>
> In this particular issue I think it's more about surprising behavior than 
> good defaults. It's surprising to me that go vet would prevent me from 
> running a test. If I don't run it now, and so it doesn't prevent me from 
> running tests, why should it in the future? It's surprising to me that 
> effectively a "linting tool" is being run automatically when I run tests. 
> It's surprising that when I compile a binary these same tests are not being 
> run, and do not prevent me from compiling the binary, but specifically 
> running tests, why is this? It's surprising to me that if I already run the 
> go vet check before tests in my project, I'm actually running it twice now 
> (in all projects that were savvy enough to already be doing this). It's 
> also surprising that it's been implemented at such a low level (directly 
> in the toolchain) when it's effectively just calling two commands in 
> succession.
>
> Is there some strong reason to have it in the toolchain which I'm not 
> seeing?
> Is there a good reason why it's on by default?
> Maybe the correct answer is to have go test -vet=on, where off is the 
> default?
>
> J

[go-nuts] [ANN] go-diff

[Markus Zimmermann]

Hi gophers!

A while ago I took over maintainership of https://github.com/sergi/go-diff 
which is a package to diff, match and patch text. The original code is a 
port of Neil Fraser's google-diff-match-patch by Sergi Mansilla. I am 
almost finished with what I wanted to do when I started this, which is 
refactoring the whole code base to make it more maintainable, and work down 
PRs and open issues.

However, there is still lots that could be done 
[https://github.com/sergi/go-diff/issues]. If somebody wants to help out 
and work on an open source project I would be happy to mentor and review 
your code.

Cheers,
Markus

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Deleting the /r/golang subreddit

[Markus Zimmermann]

Isn't this like saying a whole country is bad because of its 
president/leader? A whole community cannot be bad because of what one 
person did, nor can a whole company because of its CEO. golang-nuts/-dev 
would not be closed if one Google person behaves bad. We are a community of 
our own, we are the Go community as stated in https://golang.org/conduct 
and the CEO of Reddit is afaik not part of that community.

I am not a Reddit user, but I vote for making /r/golang unofficial, I do 
not know what that entails, and maybe transferring moderator rights to 
other people.

On Friday, November 25, 2016 at 12:53:32 AM UTC+1, bradfitz wrote:
>
> In light of the CEO of Reddit admitting to editing user comments (see 
> dozen news stories today), I propose we delete the /r/golang subreddit.
>
> That is so beyond unethical and immature, I no longer want anything to do 
> with that site. I will be deleting my account on Reddit after backing up my 
> content, and I will no longer be a moderator of /r/golang.
>
> If other moderators of /r/golang feel strongly that it should remain, I 
> suppose you're welcome to keep it going.
>
> But if the other moderators want to abandon it and focus our conversation 
> elsewhere (or build a replacement), I'm happy to just delete /r/golang.
>
> Opinions?
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: [ANN] Cgogen - automatic Go bindings generator released

[Markus Zimmermann]

On Tuesday, September 20, 2016 at 1:05:39 PM UTC+2, Maxim Kupriianov wrote:
>
> Hi Markus, nice project! I must agree that the subject-specific bindings 
> will always be superior over the generic ones. Another good example of that 
> is https://github.com/therecipe/qt bindings with custom generator as well.
>

We tried to keep a lot of code generic so that we could make something like 
cgogen later on. You beat us to it :-) I would have loved to use something 
like cgogen but could not find anything related at that time.

Wow, github.com/therecipe/qt is huge.

As for LLVM, I'm trying to avoid using it for now, because that's a very 
> heavy dependency to have. Also that'd require rewriting more than half of 
> the current code. One day we may join our efforts working on a generic C 
> code transcriber, but that's another story.
>

The dependency is not that big, it is just libclang and some other 
libraries however it is not as portable as having a C parser written in Go. 
A rewrite at this point would not make sense for either projects. Honestly, 
if something like github.com/cznic/cc had existed when I started working 
with go-clang I would not have started the whole rewrite. Are you a 
maintainer of github.com/cznic/cc ? What is your opinion of the maturity of 
the project?

> Maybe we can find some inspiration from each others projects?
>
> I find the "ArrayNameFromLength" function curious, sadly things like that 
> are almost impossible in a generic context, even with YAML hints.
>

I think that something like "ArrayNameFromLength" should be seen as a 
"first automatic draft" for a binding. If the user sees that something is 
not generated as expected, the user should be able to overwrite the 
generators heuristics. That is what we did with go-clang/gen. However, I 
agree with your concern. Such heuristics suck, but I think they are better 
than having none.

Take a look onto my helper pipeline (gen_bindings.go), I used that approach 
> instead of using templates that are pure evil for generating code.
>

We used go/ast to generate most Go code which is complicated to use. 
However, I am pretty happy with the result and after some refactoring the 
whole generation would look rather easy. I did that for an internal project 
(I hope that I can open source parts of it) and this generic approach makes 
it easy to maintain generators for more than one language.

I definitely will study your code deeply because it's interesting indeed to 
> compare our approaches to the same problems.
> Feel free to reach me out :)
>

Looking forward to your changes :-) Contact me if you need any 
ideas/opinions.

I mentioned some concerns about cgogen here 
https://github.com/go-clang/design/issues/9 if you are interested. I can 
also over my help in setting up TravisCI and Coveralls, and making the 
whole repository more "go-ish". 

On Tue, Sep 20, 2016 at 1:23 PM, Markus Zimmermann  > wrote:
>
>> This looks pretty neat. We did something similar for 
>> https://github.com/go-clang/ The generator is here 
>> https://github.com/go-clang/gen and a resulting binding is here 
>> https://github.com/go-clang/v3.7 Maybe we can find some inspiration from 
>> each others projects? It would be also interesting to figure out how we 
>> could merge each efforts?
>>
>> Cheers,
>> Markus
>>
>> On Tuesday, September 20, 2016 at 10:19:14 AM UTC+2, Maxim Kupriianov 
>> wrote:
>>>
>>> Hello everyone, 
>>>
>>> today I'm glad to announce that after 3 months of full-time development 
>>> back in 2015 and after 1 year of part-time field testing and improvements 
>>> in 2016,
>>> an automatic CGo bindings generator for Golang is finally released to 
>>> the public. Visit https://cgogen.com
>>>
>>> Sources: http://github.com/xlab/cgogen
>>> Documentation: https://github.com/xlab/cgogen/wiki
>>>
>>> That is the same generator that brought us Go bindings for Android NDK, 
>>> Vulkan Graphics API, CMU PocketSphinx, ALAC and Ogg/Vorbis decoders, Pure 
>>> Data embeddable library, PortAudio and PortMIDI adapters. And bindings for 
>>> the libpvpx from WebM are on their way.
>>>
>>> I hope the project will be useful for the community and awaiting for the 
>>> feedback and issues.
>>> Good luck y all!
>>>
>>> --
>>> Max
>>>
>> -- 
>> You received this message because you are subscribed to a topic in the 
>> Google Groups "golang-nuts" group.
>> To unsubscribe from this topic, visit 
>> https://groups.google.com/d/topic/golang-nuts/3I7TzmEirbo/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to 
>> golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: [ANN] Cgogen - automatic Go bindings generator released

[Markus Zimmermann]

This looks pretty neat. We did something similar for 
https://github.com/go-clang/ The generator is here 
https://github.com/go-clang/gen and a resulting binding is here 
https://github.com/go-clang/v3.7 Maybe we can find some inspiration from 
each others projects? It would be also interesting to figure out how we 
could merge each efforts?

Cheers,
Markus

On Tuesday, September 20, 2016 at 10:19:14 AM UTC+2, Maxim Kupriianov wrote:
>
> Hello everyone, 
>
> today I'm glad to announce that after 3 months of full-time development 
> back in 2015 and after 1 year of part-time field testing and improvements 
> in 2016,
> an automatic CGo bindings generator for Golang is finally released to the 
> public. Visit https://cgogen.com
>
> Sources: http://github.com/xlab/cgogen
> Documentation: https://github.com/xlab/cgogen/wiki
>
> That is the same generator that brought us Go bindings for Android NDK, 
> Vulkan Graphics API, CMU PocketSphinx, ALAC and Ogg/Vorbis decoders, Pure 
> Data embeddable library, PortAudio and PortMIDI adapters. And bindings for 
> the libpvpx from WebM are on their way.
>
> I hope the project will be useful for the community and awaiting for the 
> feedback and issues.
> Good luck y all!
>
> --
> Max
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Why is there no " declared and not used" compile error?

[Markus Zimmermann]

On Wednesday, September 14, 2016 at 3:37:10 PM UTC+2, Alan Donovan wrote:
>
> On 14 September 2016 at 09:32, Markus Zimmermann  > wrote:
>>
>> Do you think this might be worth bringing up in golang-dev?
>>
>
> No.  We can't make this change to the language without breaking existing 
> programs.
>

That is unfortunately. I will just write a linter check for that. Thanks 
for the talk!

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Why is there no " declared and not used" compile error?

[Markus Zimmermann]

On Wednesday, September 14, 2016 at 3:47:29 PM UTC+2, Chris Hines wrote:
>
> On Monday, September 12, 2016 at 8:57:15 AM UTC-4, Markus Zimmermann wrote:
>>
>> Hi gophers!
>>
>> I am wondering why we have a "declared and not used" compile error for 
>> variables [https://play.golang.org/p/KLzHVO5L7q] but not for constants [
>> https://play.golang.org/p/JG9dSa_VKg]? I am sure there is some rational 
>> decision behind this that I have missed, but my Google-fu is just not 
>> strong enough to find an answer.
>>
>> Cheers,
>> Markus
>>
>
> It may be a stretch, but allowing unused constants may be desirable when 
> we're using iota and we don't want higher constants to change when some of 
> the intermediate values become unused.
>

No reason to keep an unused constant for that. You can use the blank 
identifier https://play.golang.org/p/A55mWz_c93

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Why is there no " declared and not used" compile error?

[Markus Zimmermann]

On Tuesday, September 13, 2016 at 3:50:12 PM UTC+2, Alan Donovan wrote:
>
> On 13 September 2016 at 08:22, Markus Zimmermann  > wrote:
>
>> On Monday, September 12, 2016 at 3:41:35 PM UTC+2, adon...@google.com 
>> wrote:
>>>
>>> unused constants and types cost nothing at run time.  It's not that 
>>> simple, of course, because constant and type declarations may be the only 
>>> uses of an import of a package whose initialization has some cost; but this 
>>> is quite marginal.
>>>
>>
>> The const declaration of my example is inside an unexported function 
>> body. It can not be accessed by an import. So, it should really matter only 
>> at compile time, right?
>>
>
> Right.  The import case I was referring to is this:
>
> import "p"
> func main() {
>   const unused = p.K
> }
>
> The unused constant causes p to be an (unnecessary) dependency, which in 
> turn causes p.init to be executed.
>

This holds true for every declaration with an initialization, and should be 
therefore be no reason to allow such unused constant declarations. There is 
also already the official idiom to use the blank identifier as the explicit 
package name if someone really needs p.init to be executed.

However, I am referring to the definition of 
>> https://golang.org/doc/faq#unused_variables_and_imports which states 
>> "... Go refuses to compile programs with unused variables or imports, 
>> trading short-term convenience for long-term build speed and program 
>> clarity." Which holds true in the same way for an unused const as it does 
>> for an unused variable. However, even though it produces the same results, 
>> i.e. a new declaration which decreases the program clarity, only the 
>> variable declaration is marked.
>>
>
> You are right that the motivation given in the FAQ should apply equally to 
> constants and types.
>

Do you think this might be worth bringing up in golang-dev?

 Additionally, if an unused constant is compiled into the binary it does 
>> also matter to the binary size, even just a little bit.
>
>
> An unused constant is unlikely to affect the size of the final executable 
> since in a typical compiler generates code or data for a constant only when 
> it is used. It will be present in the intermediate .a files though.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Why is there no " declared and not used" compile error?

[Markus Zimmermann]

On Monday, September 12, 2016 at 3:21:50 PM UTC+2, Sjon Kosse wrote:
>
> Hello,
>
> Const values can be accesed still by reflection in a reasonable manner. 
> This question has come by quite a few times before on the mailing list, 
> better explanations should be easy to find.
>

I do not think that this is true. Wouldn't this already break the first 
reflection law? How can you examine something with the reflection package 
without mentioning it in the source code? Where would the interface value 
come from? Additionally, my example does not export anything, it is a 
constant inside an unexported function.


Cheers,
Markus

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Why is there no " declared and not used" compile error?

[Markus Zimmermann]

On Monday, September 12, 2016 at 3:41:35 PM UTC+2, adon...@google.com wrote:
>
> On Monday, 12 September 2016 08:57:15 UTC-4, Markus Zimmermann wrote:
>>
>> Hi gophers!
>>
>> I am wondering why we have a "declared and not used" compile error for 
>> variables [https://play.golang.org/p/KLzHVO5L7q] but not for constants [
>> https://play.golang.org/p/JG9dSa_VKg]? I am sure there is some rational 
>> decision behind this that I have missed, but my Google-fu is just not 
>> strong enough to find an answer.
>>
>
> I suspect the answer is: because unused constants and types cost nothing 
> at run time.  It's not that simple, of course, because constant and type 
> declarations may be the only uses of an import of a package whose 
> initialization has some cost; but this is quite marginal.
>

The const declaration of my example is inside an unexported function body. 
It can not be accessed by an import. So, it should really matter only at 
compile time, right? However, I am referring to the definition of 
https://golang.org/doc/faq#unused_variables_and_imports which states "... 
Go refuses to compile programs with unused variables or imports, trading 
short-term convenience for long-term build speed and program clarity." 
Which holds true in the same way for an unused const as it does for an 
unused variable. However, even though it produces the same results, i.e. a 
new declaration which decreases the program clarity, only the variable 
declaration is marked. Additionally, if an unused constant is compiled into 
the binary it does also matter to the binary size, even just a little bit.

Cheers,
Markus

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] What is the correct usage for the -msan flag?

[Markus Zimmermann]

On Friday, September 9, 2016 at 1:51:49 PM UTC+2, Ian Lance Taylor wrote:

> > I have the following questions concerning the usage of the -msan flag: 
> > a.) Are these the preferred arguments to use the -msan flag? 
>
> Looks OK to me. 
>

I am wondering if they are really needed, and if yes, why are they not 
included at compile time by the Go compiler? Also, would it make sense to 
add -fsanitize-memory-track-origins as -msan-track-origins flag to Go build?

> b.) Why are there no line numbers in the traces? 
>
> I don't know.  Try adding -g to CGO_CPPFLAGS to see if it makes a 
> difference. 
>
> > c.) Any advice on how I can debug such problems? 
>
> If -g doesn't help then I think you'll have to use objdump or gdb or 
> lldb to look at the code at the failing address.  Those tools should 
> at least let you map that address back to a source line.  Note that 
> the problem does not seem to be in Go code, which of course is 
> unsurprising. 
>

Unfortunately -g does not make a difference. However, you are right that 
this is not a Go problem and I will create a bug report in the LLVM tracker.

> d.) What are the official minimum requirements for the -msan flag? (LLVM 
> >= 
> > 3.8? ...?) 
>
> Yes, on GNU/Linux you need at least LLVM 3.8.  I don't know of any 
> other requirements. 
>

I am wondering if the version requirement should be added to the help text 
of -msan? It currently states "enable interoperation with memory sanitizer. 
Supported only on linux/amd64, and only with Clang/LLVM as the host C 
compiler." I am also wondering why these conditions (as stated in 
https://golang.org/misc/cgo/testsanitizers/test.bash ) are not checked 
during compile time? Would that be a good Go addition?

Thanks for your answers. They are really helpful.

Cheers,
Markus

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Why is there no " declared and not used" compile error?

[Markus Zimmermann]

Hi gophers!

I am wondering why we have a "declared and not used" compile error for 
variables [https://play.golang.org/p/KLzHVO5L7q] but not for constants 
[https://play.golang.org/p/JG9dSa_VKg]? I am sure there is some rational 
decision behind this that I have missed, but my Google-fu is just not 
strong enough to find an answer.

Cheers,
Markus

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] What is the correct usage for the -msan flag?

[Markus Zimmermann]

Hi gophers!

I am trying to integrate -msan for testing into 
https://github.com/go-clang/bootstrap using Go 1.7.1 and LLVM 3.9.0 inside 
an Ubuntu Trusty 64bit VM. The Kernel is 3.13.0.

I am executing a test using the following command:

CC=clang CGO_LDFLAGS="-L`llvm-config --libdir` -fsanitize=memory" 
> CGO_CPPFLAGS='-fsanitize=memory -fsanitize-memory-track-origins 
> -fno-omit-frame-pointer' go test -timeout 60s -v -msan 
> -test.run=TestBasicParsing
>

And the address sanitizer detects a problem for this test giving me the 
following output:

Uninitialized bytes in __interceptor_memchr at offset 6 inside 
> [0x7080bf58, 7)
> ==32156==WARNING: MemorySanitizer: use-of-uninitialized-value
> #0 0x7fdbcc7e3c85  
> (/usr/lib/x86_64-linux-gnu/libLLVM-3.9.so.1+0x653c85)
> #1 0x7fdbcc7f40a7  
> (/usr/lib/x86_64-linux-gnu/libLLVM-3.9.so.1+0x6640a7)
> #2 0x7fdbd0723bbe  
> (/usr/lib/x86_64-linux-gnu/libclang-3.9.so.1+0xc05bbe)
> #3 0x7fdbd07334ca  
> (/usr/lib/x86_64-linux-gnu/libclang-3.9.so.1+0xc154ca)
> #4 0x7fdbcffaef39  
> (/usr/lib/x86_64-linux-gnu/libclang-3.9.so.1+0x490f39)
> #5 0x7fdbcff8c504  
> (/usr/lib/x86_64-linux-gnu/libclang-3.9.so.1+0x46e504)
> #6 0x7fdbcfd8178e  
> (/usr/lib/x86_64-linux-gnu/libclang-3.9.so.1+0x26378e)
> #7 0x7fdbcc7be314  
> (/usr/lib/x86_64-linux-gnu/libLLVM-3.9.so.1+0x62e314)
> #8 0x7fdbcc7be393  
> (/usr/lib/x86_64-linux-gnu/libLLVM-3.9.so.1+0x62e393)
> #9 0x7fdbcc8214bc  
> (/usr/lib/x86_64-linux-gnu/libLLVM-3.9.so.1+0x6914bc)
> #10 0x7fdbcf908183  (/lib/x86_64-linux-gnu/libpthread.so.0+0x8183)
> #11 0x7fdbced0d37c  (/lib/x86_64-linux-gnu/libc.so.6+0xfa37c)
>
>   Uninitialized value was stored to memory at
> #0 0x42d0d5  
> (/tmp/go-build816629300/github.com/go-clang/bootstrap/clang/_test/clang.test+0x42d0d5)
> #1 0x7fdbcbf479bd  (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0xbb9bd)
>
>   Uninitialized value was stored to memory at
> #0 0x42d0d5  
> (/tmp/go-build816629300/github.com/go-clang/bootstrap/clang/_test/clang.test+0x42d0d5)
> #1 0x7fdbcbf46e2f  (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0xbae2f)
>
>   Uninitialized value was created by a heap allocation
> #0 0x42b6d9  
> (/tmp/go-build816629300/github.com/go-clang/bootstrap/clang/_test/clang.test+0x42b6d9)
> #1 0x7fdbcbeeadac  (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0x5edac)
>
> SUMMARY: MemorySanitizer: use-of-uninitialized-value 
> (/usr/lib/x86_64-linux-gnu/libLLVM-3.9.so.1+0x653c85)
>


I have the following questions concerning the usage of the -msan flag:
a.) Are these the preferred arguments to use the -msan flag?
b.) Why are there no line numbers in the traces?
c.) Any advice on how I can debug such problems?
d.) What are the official minimum requirements for the -msan flag? (LLVM >= 
3.8? ...?)

I appreciate any input since I am completely lost on how to address this 
problem :-/

Cheers,
Markus

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: ANNOUNCE: gini

[Markus Zimmermann]

>   ★ / 
>
>   
>   ★   
>
>   
> / 
>
> /  
>  
>
>   /
>  
>
> /  
>  
>
>   /
>  
>
> /  
>  
>
>   /
>  
>
> /  
>  
>
>   /
>  
>
> /  
>  
>
>   /
>  
>
> /  
>  
>
>   /
>  
>
> /  
>  
>
>   /
>  
>
> /  
>  
>
>   /
>  
>
> /  
>  
>
>   /
>  
>
> /  
>  
>
>   /
>  
>
> /  
>  
>
>   /
>  
>
> /  
>  
>
>   / ☆   ☆  
>  
>
> /  
>  
>
>   /
>  
>
> /  
>  
>
>   / ☆  
>  
>
> /  
>  
>
>   / ☆ 
>   ☆   
>
> / ☆   ☆
>  
>
>   / ☆  
>  
>
>   ★ /  
>  
>
>   /
>  
>
> ★   ☆   ☆  
>  
>
> ★ / ☆ ☆
>  
>
> ★  
>  
>
>     
>
>
> 
>
> ★ - ps wins
>
> ☆ - g768l wins
>
>
> This is SAT, it is a hard and heuristic problem.  Your results may vary, 
> and certainly there exists problems where other solvers are better (this is 
> true for every solver).
>
> We are in the process of drafting/submitting some papers related to gini.
>
> Enjoy,
> Scott
>
> Le dimanche 24 juillet 2016 12:55:39 UTC+2, Markus Zimmermann a écrit :
>>
>> I also would like to see some examples. Also, please make the whole 
>> project go-getable.
>>
>> Looking forward to see where this project is going, exciting! Can you 
>> link to the papers where you demonstrate that you outperform picosat and 
>> minisat?
>>
>> On Saturday, July 23, 2016 at 11:33:31 PM UTC+2, Scott Cotton wrote:
>>>
>>> I'm happy to announce the first public beta release of mini, available 
>>> at 
>>> github <http://github.com/IRIFrance/gini>.
>>>
>>> Gini is a SAT solver with some related tools built for solving the 
>>> canonical NP-complete SAT problem.  SAT solvers have many applications in 
>>> formal verification and discrete optimisation,
>>> often acting as an indispensable component in these domains.
>>>
>>> Gini is written in 100% pure go and thus far, our core CDCL solver 
>>> either outperforms or is competitive with analogs in C/C++ like picosat and 
>>> minisat.  Additionally, internal measures of raw speed such as 
>>> mega-props/second  are good and independent of variations arising from 
>>>  heuristics.
>>>
>>> By bringing a high quality SAT solver to go, we hope to enable 
>>> competitive innovations in the go community which tackle combinatorial 
>>> explosion symbolically.
>>>
>>> Gini is in first beta public release, following the recent SAT 
>>> competition.  To maintain performance in the long term, we plan to have 
>>> gini compete in sat races and sat competitions annually. To this end, we 
>>> are happy to collaborate with gophers, the curious, raw speed junkies, 
>>> algorithm officianados, and logicians alike.  
>>>
>>> Cheers,
>>>
>>>
>>> -- 
>>> Scott Cotton
>>> President, IRI France SAS
>>> http://www.iri-labs.com
>>>
>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: How to capture stderr/stdout of cgo calls?

[Markus Zimmermann]

The documentaton says that freopen closes the original stream. Wouldn't 
that mean, that it is not possible to restore the origianl stderr/stdout?

On Friday, August 5, 2016 at 4:52:57 AM UTC+2, mattn wrote:
>
> I guess calling freopen is better because cgo possibly having reference of 
> stdin/stdout as FILE*.
>
>
> On Thursday, August 4, 2016 at 8:07:40 PM UTC+9, Markus Zimmermann wrote:
>>
>> I feared that there would be no pure Go solution. Anyway, I used fdopen 
>> variant the code is here 
>> https://github.com/zimmski/osutil/blob/master/capture.go#L51 if anyone 
>> every stumbles over this thread.
>>
>> Thanks for the help.
>>
>> On Monday, July 25, 2016 at 1:00:46 PM UTC+2, Uli Kunitz wrote:
>>>
>>> Your program doesn't work because changing os.Stdout and os.Stdin has no 
>>> effect on stdio stdin and stdout. Stdio stdout will still reference fd 1 
>>> and stderr fd 2.
>>>  
>>> I recommend following three solutions:
>>>
>>>
>>>1. The simplest approach could be not to use cgo at all and run the 
>>>C code in a separate process. Capturing the output of a separate process 
>>> is 
>>>simple and can be done using the os/exec package.
>>>2. Reset stdin and stdout using fdopen in the C code. This could be 
>>>done permanently.
>>>3. Swap file descriptors using dup2.
>>>
>>>  
>>>  
>>>  
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: How to capture stderr/stdout of cgo calls?

[Markus Zimmermann]

You are right Sebastien, I fixed that. Thanks!

On Thursday, August 4, 2016 at 6:58:06 PM UTC+2, Sebastien Binet wrote:
>
>
>
> On Thu, Aug 4, 2016 at 1:07 PM, Markus Zimmermann  > wrote:
>
>> I feared that there would be no pure Go solution. Anyway, I used fdopen 
>> variant the code is here 
>> https://github.com/zimmski/osutil/blob/master/capture.go#L51 if anyone 
>> every stumbles over this thread.
>>
>
> seems to me like you're leaking C.CString("w") at:
> https://github.com/zimmski/osutil/blob/master/capture.go#L64
>
> it's allocated from cgo so you need to:
> cw := C.CString("w")
> defer C.free(unsafe.Pointer(cw))
> // ...
>
> -s
>  
>
>>
>> Thanks for the help.
>>
>>
>> On Monday, July 25, 2016 at 1:00:46 PM UTC+2, Uli Kunitz wrote:
>>>
>>> Your program doesn't work because changing os.Stdout and os.Stdin has no 
>>> effect on stdio stdin and stdout. Stdio stdout will still reference fd 1 
>>> and stderr fd 2.
>>>  
>>> I recommend following three solutions:
>>>
>>>
>>>1. The simplest approach could be not to use cgo at all and run the 
>>>C code in a separate process. Capturing the output of a separate process 
>>> is 
>>>simple and can be done using the os/exec package.
>>>2. Reset stdin and stdout using fdopen in the C code. This could be 
>>>done permanently.
>>>3. Swap file descriptors using dup2.
>>>
>>>  
>>>  
>>>  
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: How to capture stderr/stdout of cgo calls?

[Markus Zimmermann]

I feared that there would be no pure Go solution. Anyway, I used fdopen 
variant the code is here 
https://github.com/zimmski/osutil/blob/master/capture.go#L51 if anyone 
every stumbles over this thread.

Thanks for the help.

On Monday, July 25, 2016 at 1:00:46 PM UTC+2, Uli Kunitz wrote:
>
> Your program doesn't work because changing os.Stdout and os.Stdin has no 
> effect on stdio stdin and stdout. Stdio stdout will still reference fd 1 
> and stderr fd 2.
>  
> I recommend following three solutions:
>
>
>1. The simplest approach could be not to use cgo at all and run the C 
>code in a separate process. Capturing the output of a separate process is 
>simple and can be done using the os/exec package.
>2. Reset stdin and stdout using fdopen in the C code. This could be 
>done permanently.
>3. Swap file descriptors using dup2.
>
>  
>  
>  
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] How to capture stderr/stdout of cgo calls?

[Markus Zimmermann]

Given the following code https://play.golang.org/p/1_Q9SFIPjr one can see 
that stdout of Go and cgo are piped to two different descriptors.

a.) Is it possible to redirect stderr/stdout of cgo calls directly to 
os.Stderr/os.Stdout? Am I overlooking some argument or options of cgo so 
that is done automatically during compilation?
b.) Are there any hacks to accomplish the capturing involving only Go code?
c.) Is there a solution at all?

Either my Google-foo is not strong enough or nobody ever posted this on the 
mailing list or some blog.

Cheers,
Markus

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: ANNOUNCE: gini

[Markus Zimmermann]

I also would like to see some examples. Also, please make the whole project 
go-getable.

Looking forward to see where this project is going, exciting! Can you link 
to the papers where you demonstrate that you outperform picosat and minisat?

On Saturday, July 23, 2016 at 11:33:31 PM UTC+2, Scott Cotton wrote:
>
> I'm happy to announce the first public beta release of mini, available at 
> github .
>
> Gini is a SAT solver with some related tools built for solving the 
> canonical NP-complete SAT problem.  SAT solvers have many applications in 
> formal verification and discrete optimisation,
> often acting as an indispensable component in these domains.
>
> Gini is written in 100% pure go and thus far, our core CDCL solver either 
> outperforms or is competitive with analogs in C/C++ like picosat and 
> minisat.  Additionally, internal measures of raw speed such as 
> mega-props/second  are good and independent of variations arising from 
>  heuristics.
>
> By bringing a high quality SAT solver to go, we hope to enable competitive 
> innovations in the go community which tackle combinatorial explosion 
> symbolically.
>
> Gini is in first beta public release, following the recent SAT 
> competition.  To maintain performance in the long term, we plan to have 
> gini compete in sat races and sat competitions annually. To this end, we 
> are happy to collaborate with gophers, the curious, raw speed junkies, 
> algorithm officianados, and logicians alike.  
>
> Cheers,
>
>
> -- 
> Scott Cotton
> President, IRI France SAS
> http://www.iri-labs.com
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] [ANN] go-mutesting v1.0 - Mutation testing for Go source code

[Markus Zimmermann]

Hi gophers!

I would like to announce v1.0 of go-mutesting 
https://github.com/zimmski/go-mutesting

The following has changed since the last release:
- Added a build-in exec command so only the go-mutation binary is needed 
for mutation testing in the most common use-case.
- Added a mutator for "case" clauses.
- Included noop-statements to avoid "declared but never used" compile 
errors.
- Additional guards to avoid compile errors.
- Added the "--match" argument to apply mutation testing only on certain 
functions.
- Added "test-current-directory.sh" exec command to run tests in the 
current directory for the mutation.
- Added "test-mutated-package.sh" exec command to run tests in the mutated 
file's package.
- Completely changed the interface and internal behavior of mutators. It is 
now a lot easier to write mutators.
- Added more tests.
- Upgraded to Go 1.6 for the development environment.
- Cleaned up the development environment.
- Took a good look at other mutation testing tools and included some of 
their features for future releases.

As always, I appreciate any suggestions and comments. There is still a lot 
that could be done (especially reducing bad substitution that lead to 
compile errors) but I have a lot on my plate right now. If some new-comer 
or eager gophers wants to contribute, I would be very happy to help out 
giving guidance. There are some ideas for nice additions in the issue 
tracker https://github.com/zimmski/go-mutesting/issues which are partly 
easy to implement.

Cheers,
Markus

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.