Re: [Cocci] More support for SmPL data processing with databases?
> I think that you just want to disable the isomorphism. Put disable neg_if > in the initial @@ of your rule. >> >> Will a need evolve to switch isomorphisms completely off for safer >> (and efficient) analysis of original source code? Will the corresponding system configuration get any further development considerations? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
On Mon, 13 Apr 2020, Markus Elfring wrote: > >> I would find it nice to explain the different software behaviour for the > >> mentioned > >> SmPL code variants > > > > Run spatch -parse-cocci and you will easily see. The if stays in the same > > place. When the isomorphism exchanges the branches, the condition gets > > negated, so in one case you get the position of the full condition and > > in the other case you get the position of the part of the condition under > > the negation. > > Do any of these transformations avoid the reporting of duplicate source > code positions? Scripts are run for each set of metavariable bindings. But there is no special treatment of positions. julia > > > >>> I think that you just want to disable the isomorphism. Put disable neg_if > >>> in the initial @@ of your rule. > > Will a need evolve to switch isomorphisms completely off for safer > (and efficient) analysis of original source code? > > Regards, > Markus > ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
>> I would find it nice to explain the different software behaviour for the >> mentioned >> SmPL code variants > > Run spatch -parse-cocci and you will easily see. The if stays in the same > place. When the isomorphism exchanges the branches, the condition gets > negated, so in one case you get the position of the full condition and > in the other case you get the position of the part of the condition under > the negation. Do any of these transformations avoid the reporting of duplicate source code positions? >>> I think that you just want to disable the isomorphism. Put disable neg_if >>> in the initial @@ of your rule. Will a need evolve to switch isomorphisms completely off for safer (and efficient) analysis of original source code? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
On Mon, 13 Apr 2020, Markus Elfring wrote: > >>> Maybe you consider the following results to be a problem: > >>> > >>> kzalloc mcde [("mcde_probe", 2, 0, 198, 1, "m.c", 16, 1, 16, 3)] > >>> kzalloc ! mcde [("mcde_probe", 2, 0, 198, 1, "m.c", 16, 1, 16, 3)] > >>> platform_get_irq irq [("mcde_probe", 2, 0, 198, 1, "m.c", 97, 1, 97, 3)] > >>> platform_get_irq ! irq [("mcde_probe", 2, 0, 198, 1, "m.c", 97, 1, 97, 3)] > >>> > >>> The positions are the same. The result is because of an isomorphism that > >>> exchanges the branches of a conditional. > > I would find it nice to explain the different software behaviour for the > mentioned > SmPL code variants Run spatch -parse-cocci and you will easily see. The if stays in the same place. When the isomorphism exchanges the branches, the condition gets negated, so in one case you get the position of the full condition and in the other case you get the position of the part of the condition under the negation. julia > > A: > if@p ( > ( <+... result ...+> > & check > ) ) > > > B: > if ( > ( <+... result ...+> > & check@p > ) ) > > > > >> Can the data which are provided because of isomorphism transformations > >> be distinguished by an additional attribute? > > > > I think that you just want to disable the isomorphism. Put disable neg_if > > in the initial @@ of your rule. > > Thanks for this suggestion. > > Source code analysis can be continued also with this configuration option. > > > Now I become also more interested in improvements around parallel data > processing. > Which data structures would support aggregate computations for a growing code > base? > > Regards, > Markus > ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
>>> Maybe you consider the following results to be a problem: >>> >>> kzalloc mcde [("mcde_probe", 2, 0, 198, 1, "m.c", 16, 1, 16, 3)] >>> kzalloc ! mcde [("mcde_probe", 2, 0, 198, 1, "m.c", 16, 1, 16, 3)] >>> platform_get_irq irq [("mcde_probe", 2, 0, 198, 1, "m.c", 97, 1, 97, 3)] >>> platform_get_irq ! irq [("mcde_probe", 2, 0, 198, 1, "m.c", 97, 1, 97, 3)] >>> >>> The positions are the same. The result is because of an isomorphism that >>> exchanges the branches of a conditional. I would find it nice to explain the different software behaviour for the mentioned SmPL code variants A: if@p ( ( <+... result ...+> & check ) ) B: if ( ( <+... result ...+> & check@p ) ) >> Can the data which are provided because of isomorphism transformations >> be distinguished by an additional attribute? > > I think that you just want to disable the isomorphism. Put disable neg_if > in the initial @@ of your rule. Thanks for this suggestion. Source code analysis can be continued also with this configuration option. Now I become also more interested in improvements around parallel data processing. Which data structures would support aggregate computations for a growing code base? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
On Mon, 13 Apr 2020, Markus Elfring wrote: > > Maybe you consider the following results to be a problem: > > > > kzalloc mcde [("mcde_probe", 2, 0, 198, 1, "m.c", 16, 1, 16, 3)] > > kzalloc ! mcde [("mcde_probe", 2, 0, 198, 1, "m.c", 16, 1, 16, 3)] > > platform_get_irq irq [("mcde_probe", 2, 0, 198, 1, "m.c", 97, 1, 97, 3)] > > platform_get_irq ! irq [("mcde_probe", 2, 0, 198, 1, "m.c", 97, 1, 97, 3)] > > > > The positions are the same. The result is because of an isomorphism that > > exchanges the branches of a conditional. > > Can the data which are provided because of isomorphism transformations > be distinguished by an additional attribute? I think that you just want to disable the isomorphism. Put disable neg_if in the initial @@ of your rule. julia > > > > I think that you could have figured this out by yourself. If you just > > printed out the data instead of wrapping it in lots of database noise, > > then you could have seen what was going on. > > I got the impression that there are further software development > challenges to consider (also for safer counting of record sets). > > Regards, > Markus > ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
> Maybe you consider the following results to be a problem: > > kzalloc mcde [("mcde_probe", 2, 0, 198, 1, "m.c", 16, 1, 16, 3)] > kzalloc ! mcde [("mcde_probe", 2, 0, 198, 1, "m.c", 16, 1, 16, 3)] > platform_get_irq irq [("mcde_probe", 2, 0, 198, 1, "m.c", 97, 1, 97, 3)] > platform_get_irq ! irq [("mcde_probe", 2, 0, 198, 1, "m.c", 97, 1, 97, 3)] > > The positions are the same. The result is because of an isomorphism that > exchanges the branches of a conditional. Can the data which are provided because of isomorphism transformations be distinguished by an additional attribute? > I think that you could have figured this out by yourself. If you just > printed out the data instead of wrapping it in lots of database noise, > then you could have seen what was going on. I got the impression that there are further software development challenges to consider (also for safer counting of record sets). Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
> @script:ocaml@ > action << find.action; > check << find.check; > p << find.p; > @@ > Printf.printf "%s %s %s\n" action check (Dumper.dump p) > > On the code you provided whichis unfortunately much too long to include in > a message I found my code examples short enough. > and that you could have surely shortened to illustrate whatever > is the problem. Does such a feedback indicate that you are still struggling with the understanding for the presented software situation? > I get the same output from both semantic patches, except that the line > numbers are different for the position variable p, Does this information contain a contradiction? Do you observe any repeated data sets? > because you have put it in a different place. Yes. - Thus I would expect that different column numbers will be displayed while the line numbers should be (mostly?) identical. > I don't see any reason why making the script code call a database would > have any impact on this issue. The database can provide ACID properties for parallel data processing, can't it? I would like to perform aggregate computations finally. Would you become interested in counting record sets anyhow? > If you do get different results for the above semantic patches, > please make a small .c file that illustrates the problem. I presented questionable differences once more a moment ago. https://lore.kernel.org/cocci/7228efff-76bb-44c9-e2cd-ea19bbd49...@web.de/ https://systeme.lip6.fr/pipermail/cocci/2020-April/007142.html Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
I tried the following semantic patches: @find@ expression action, check, result; position p; statement is, es; @@ result = action(...); if ( ( <+... result ...+> & check@p ) ) is else es @script:ocaml@ action << find.action; check << find.check; p << find.p; @@ Printf.printf "%s %s %s\n" action check (Dumper.dump p) @find@ expression action, check, result; position p; statement is, es; @@ result = action(...); if@p ( ( <+... result ...+> & check ) ) is else es @script:ocaml@ action << find.action; check << find.check; p << find.p; @@ Printf.printf "%s %s %s\n" action check (Dumper.dump p) On the code you provided whichis unfortunately much too long to include in a message and that you could have surely shortened to illustrate whatever is the problem. I get the same output from both semantic patches, except that the line numbers are different for the position variable p, because you have put it in a different place. I don't see any reason why making the script code call a database would have any impact on this issue. If you do get different results for the above semantic patches, please make a small .c file that illustrates the problem. The .c file should be such that it illustrates the problem and no smaller file also illustrates the problem. julia ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
>> How will the application of SmPL conjunctions evolve further? > > I really have no idea what you are talking about. I suggest to take another look at affected implementation details. > Make one single mail that contains all of the semantic patch variants > that you want to have considered and explain what the problem is. Would you find the following data processing approach easier to clarify? I extracted a bit of source code from a known file for your convenience as an attachment. https://elixir.bootlin.com/linux/v5.6.3/source/drivers/gpu/drm/mcde/mcde_drv.c#L307 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/gpu/drm/mcde/mcde_drv.c?id=8f3d9f354286745c751374f5f1fcafee6b3f3136#n308 > These semantic patches should involve no use of databases. A Python dictionary can also be used as a data storage technique for the discussed system test. @initialize:python@ @@ import sys records = {} class integrity_error: pass def store_positions(places, check_in, action_in): """Add source code positions to an internal table.""" for place in places: key = (place.file, place.line, int(place.column) + 1) if key in records: sys.stderr.write("\n".join(["-> duplicate data", "file:", key[0], "function:", place.current_element, "line:", str(place.line)])) sys.stderr.write("\n") raise integrity_error else: records[key] = (action_in, check_in, place.current_element) @find@ expression action, check, result; position p; statement is, es; @@ result = action(...); if@p ( ( <+... result ...+> & check ) ) is else es @script:python collection@ c << find.check; action << find.action; places << find.p; @@ store_positions(places, c, action) @finalize:python@ @@ if len(records) > 0: delimiter = "|" sys.stdout.write(delimiter.join(['action', 'check', '"source file"', 'line', 'column' ])) sys.stdout.write("\r\n") for key, value in records.items(): sys.stdout.write(delimiter.join([value[0], value[1], key[0], key[1], str(key[2]) ])) sys.stdout.write("\r\n") else: sys.stderr.write("No result for this analysis!\n") elfring@Sonne:~/Projekte/Coccinelle/janitor> spatch list_condition_checks_after_function_calls7.cocci ../Probe/mcde_drv-excerpt1.c … -> duplicate data file: ../Probe/mcde_drv-excerpt1.c function: mcde_probe line: 16 Traceback …: …, line 30, in store_positions __main__.integrity_error: <__main__.integrity_error instance at 0x7fd3bff4d730> … Would you like compare the software behaviour any further with the following rule variant for the semantic patch language? @find@ expression action, check, result; position p; statement is, es; @@ result = action(...); if ( ( <+... result ...+> & check@p ) ) is else es Regards, Markus // SPDX-License-Identifier: GPL-2.0 static int mcde_probe(struct platform_device *pdev) { struct device *dev = >dev; struct drm_device *drm; struct mcde *mcde; struct component_match *match = NULL; struct resource *res; u32 pid; u32 val; int irq; int ret; int i; mcde = kzalloc(sizeof(*mcde), GFP_KERNEL); if (!mcde) return -ENOMEM; mcde->dev = dev; ret = drm_dev_init(>drm, _drm_driver, dev); if (ret) { kfree(mcde); return ret; } drm = >drm; drm->dev_private = mcde; platform_set_drvdata(pdev, drm); /* Enable continuous updates: this is what Linux' framebuffer expects */ mcde->oneshot_mode = false; drm->dev_private = mcde; /* First obtain and turn on the main power */ mcde->epod = devm_regulator_get(dev, "epod"); if (IS_ERR(mcde->epod)) { ret = PTR_ERR(mcde->epod); dev_err(dev, "can't get EPOD regulator\n"); goto dev_unref; } ret = regulator_enable(mcde->epod); if (ret) { dev_err(dev, "can't enable EPOD regulator\n"); goto dev_unref; } mcde->vana = devm_regulator_get(dev, "vana"); if (IS_ERR(mcde->vana)) { ret = PTR_ERR(mcde->vana); dev_err(dev, "can't get VANA regulator\n"); goto regulator_epod_off; } ret = regulator_enable(mcde->vana); if (ret) { dev_err(dev, "can't enable VANA regulator\n"); goto regulator_epod_off; } /* * The vendor code uses ESRAM (onchip RAM) and need to activate * the v-esram34 regulator, but we don't use that yet */ /* Clock the silicon so we can access the registers */ mcde->mcde_clk = devm_clk_get(dev, "mcde"); if (IS_ERR(mcde->mcde_clk)) { dev_err(dev, "unable to get MCDE main clock\n"); ret = PTR_ERR(mcde->mcde_clk); goto regulator_off;
Re: [Cocci] More support for SmPL data processing with databases?
>> Do I present another advanced use case for the application of known software >> tools? > > No. Such a view can be also interesting. > The faults in linux paper (work done 10 years ago) uses a database as well. Thanks for this background information. How will the software evolve further in affected areas? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
> Now you have the two scripts, which is good, but you haven't included the > test data that causes the problem. Please make one mail that includes > everything. I find that I provided relevant data already. https://systeme.lip6.fr/pipermail/cocci/2020-April/007135.html https://lore.kernel.org/cocci/ab6d402e-3c12-25ab-162a-f200c382d...@web.de/ Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
>>> The database itself doesn't bring any value. >> >> Another unique constraint violation was noticed by using such technology. > > It's an accident. This might be. - I got into the development mood to publish patches for a few source files of Linux modules according to another search approach. I picked one of these files up for the discussed test. > You could have noticed the same thing by just printing the results > in the python script. This can be also possible in principle. I find the programming convenience different. >> I am curious when you are going to take related functionality better into >> account. > > Never. Interesting … > It has nothing to do with Coccinelle. Do I present another advanced use case for the application of known software tools? > When you make a bug report, you should show the minimal code that shows the > problem. I hope that variations can occasionally help also to achieve a better understanding. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
>> I observe that the following SmPL code variant can result also in >> a significant difference. >> >> @find@ >> expression action, check, result; >> position p; >> statement is, es; >> @@ >> result = action(...); >> if ( >> ( <+... result ...+> >> & check@p >> ) ) >> is >> else >> es >> >> >> How will the application of SmPL conjunctions evolve further? > > I really have no idea what you are talking about. I suggest to take another look for the software behaviour. > that contains all of the semantic patch variants that you want to have > These semantic patches should involve no use of databases. Please compare details with the following SmPL code example. @find@ expression action, check, result; position p; statement is, es; @@ result = action(...); if@p ( ( <+... result ...+> & check ) ) is else es Under which circumstances will the Coccinelle software provide duplicate data for their processing by (Python) script rules? > The database itself doesn't bring any value. Another unique constraint violation was noticed by using such technology. I am curious when you are going to take related functionality better into account. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
> I hope that it can become easier to clarify where unexpected duplicate keys > would occur as in my test approach. There is the possibility to apply information according to a clarification request on a topic like “Checking the handling of unique keys/indexes” from 2014-12-20. https://groups.google.com/d/msg/sqlalchemy/klmUwiirIQw/g6NLiVAGGAIJ elfring@Sonne:~/Projekte/Linux/next-patched> spatch ~/Projekte/Coccinelle/janitor/list_condition_checks_after_function_calls2c.cocci drivers/gpu/drm/mcde/mcde_drv.c … Using SQLAlchemy version: 1.3.15 … %% function: mcde_probe line: 322 %% function: mcde_probe line: 403 action|check|"source file"|line|column … How should the error reporting be clarified around such source code places? https://elixir.bootlin.com/linux/v5.6.3/source/drivers/gpu/drm/mcde/mcde_drv.c#L307 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/gpu/drm/mcde/mcde_drv.c?id=8f3d9f354286745c751374f5f1fcafee6b3f3136#n308 I observe that the following SmPL code variant can result also in a significant difference. @find@ expression action, check, result; position p; statement is, es; @@ result = action(...); if ( ( <+... result ...+> & check@p ) ) is else es How will the application of SmPL conjunctions evolve further? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci