Re: SonarCloud Code Smell Reduction?

[Kilo, Olabusayo]

Hello Helge (and Jon),

I looked through the changelog for the quality profiles we use in Daffodil, and 
we did indeed make changes to the configuration between April 17 and the 27th. 
We were working on configuring Sonarcloud for the project and the whole period 
from Feb till end of Apr was a spin-up period, where we were figuring things 
out. After discussing the configurations available from the default Scala and 
Java configurations, we noted which were appropriate to the project and 
enabled/disabled them in the configuration. Some of that work was detailed at 
[1] and [2]. As far for code updates addressing the issues, see [3] - [6] for 
some additional commits from that time.


[1] https://issues.apache.org/jira/browse/DAFFODIL-2272

[2] https://issues.apache.org/jira/browse/DAFFODIL-2275

[3] 
https://github.com/apache/daffodil/commit/5e63af1f5e0cd268b84e9de9735baac19310aeff

[4] 
https://github.com/apache/daffodil/commit/7be19a16eaf6a50cb22dd03f75ac16eee208b663

[5] 
https://github.com/apache/daffodil/commit/ca9353ffea9a9edf3bd266ec2d0a966b0849a387

[6] 
https://github.com/apache/daffodil/commit/5acd1777e88245626cea77ea8ce3e7df80c0fdad


--

Best Regards,

Lola K.


From: Interrante, John A (GE Research, US) 
Sent: Thursday, June 17, 2021 10:58 AM
To: dev@daffodil.apache.org 
Subject: RE: SonarCloud Code Smell Reduction?

Hello Helge,

I looked at both commits [1] and pull requests [2].  I found no commits or pull 
requests that plausibly could explain the drop in code smells from March 26 to 
April 27 on the chart [3] you linked to.  No one had mentioned doing anything 
to reduce SonarQube code smells on the dev or users lists either.  I think you 
are correct that the configuration of SonarCloud's quality profiles must have 
changed, although I don't know if any other Daffodil maintainer made a change 
or if SonarCloud made a change itself.

Mike and Steve, did you change anything in the SonarCloud configuration either?

John

[1] https://github.com/apache/daffodil/commits/master
[2] https://github.com/apache/daffodil/pulls?q=is%3Apr+is%3Aclosed
[3] 
https://sonarcloud.io/project/activity?id=apache-daffodil_date=2020-04-24T17%3A05%3A46%2B

-Original Message-
From: Helge Pfeiffer 
Sent: Wednesday, June 16, 2021 5:38 PM
To: dev@daffodil.apache.org
Subject: EXT: SonarCloud Code Smell Reduction?

Dear Daffodil developers,

My name is Helge, I am a researcher at IT University of Copenhagen [1].
I am currently conducting a study on the impact of continuous code quality 
assessment tools (SonarQube) on defects.

I am writing to you -the Daffodil developers-, since I found that Daffodil uses 
SonarCloud for continuous code quality assessment, that it is the ASF project 
with the lowest amount of code smells, vulnerabilities and 'bugs' in 
SonarCloud, and that it is the project with the biggest reduction of code 
smells (drop of ca. 60% within around 2 months) [2].

However, I am wondering if the drastic reduction of code smells that are 
reported by SonarCloud is due to code changes that address these issues or if 
it is caused by configuration of the rules ("quality profile") that SonarCloud 
applies.

I believe the latter is the case. I can only find 5 commits that are related to 
SonarQube/-Cloud or any of the reported code smells, vulnerabilities, or bugs 
[3]. I identified these commits by searching for `[Ss]mell`, 
`[Vv]ulnerabilit[iy]`, `[Bb]ug`, and `[Ss]onar` in the commit history and the 
Jira issue tracker.
I cannot see that these 5 commits are addressing multiple hundreds of 
SonarCloud code smells. However, I can see many changes of the kind `Quality 
Profile:Changes in 'Sonar way'` especially in the beginning of SonarClouds 
project activity [4].
Another possibility is of course that I just do not find the commits that 
address the SonarCloud code smells.

Therefore, I would really appreciate your feedback to my question:

  1) Is the drastic reduction of code smells that SonarCloud reports for 
Daffodil due to configuration of SonarCloud's quality profiles?
1.1) If not, could you please point me to some of the commits that address 
code smells and that I fail to identify?



Thank you in advance for your feedback and consideration. I will share the 
results of my work with you as soon they are written down in a presentable 
format.


Best regards,
Helge



---

[1] https://www.itu.dk/people/ropf/ and 
https://www.researchgate.net/profile/Helge-Pfeiffer-2
[2] https://sonarcloud.io/project/activity?id=apache-daffodil
[3] Commits related to SonarQube/-Cloud, code smells, vulnerabilities, or bugs
  * 
https://github.com/apache/daffodil/commit/2426e7f8527c289937506178a0e65da421d999ea
  * 
https://github.com/apache/daffodil/commit/f3eee732f1f5535d0177877720c4fe9f39bc3327
  * 
https://github.com/apache/daffodil/commit/075ed018d786d332deddc5e20169939f95470fef
  * 

Re: test cases - examples

[Beckerle, Mike]

Depends on what you mean by available.

They're in the daffodil source tree.

The daffodil-test module is a big suite of tests, organized by section numbers 
of the DFDL spec,
all expressed as TDML.

https://github.com/apache/daffodil/tree/master/daffodil-test/src/test/resources/org/apache/daffodil

Unfortunately, the section numbers evolved and no longer match the final DFDL 
spec., but most are the same.




From: Attila Horvath 
Sent: Thursday, June 17, 2021 1:30 PM
To: dev@daffodil.apache.org 
Subject: test cases - examples

ALCON

I assume there is a repository of test cases with which to validate and
regression test various corresponding daffodil features.

Are these test cases available to users for reference as examples?

Thx

Attila

test cases - examples

[Attila Horvath]

ALCON

I assume there is a repository of test cases with which to validate and
regression test various corresponding daffodil features.

Are these test cases available to users for reference as examples?

Thx

Attila

transformation example

[Beckerle, Mike]

DFDL isn't normally thought of as a data transformation language.

Yet it has computed elements, hidden groups, expressions that can refer to 
elements in arrays using the index position within the current array (via the 
dfdl:occursIndex() function).

These result in it having substantial data transformation capabilities.

For a while I have said that I bet one can invert a matrix in DFDL.

So I finally created an example that does so.  It takes a representation of 
data as a pair of lists, and creates a logical infoset that is a list of pairs.

https://github.com/OpenDFDL/examples/tree/master/pairsTransform

The conclusion of this little experiment is that while this is possible for 
parsing, you can't invert the process perfectly in unparsing due to some DFDL 
v1.0 restrictions. I may do experiments in Daffodil to lift those restrictions.

The notion of an entirely schema-based transform language is very interesting 
given that typical XML transformation languages such as XSLT and XQuery are 
both template/instance-document based, not schema based.




Mike Beckerle | Principal Engineer

[cid:eb186e2a-a515-4148-a5af-3d5f693737bd]

mbecke...@owlcyberdefense.com

P +1-781-330-0412

RE: SonarCloud Code Smell Reduction?

[Interrante, John A (GE Research, US)]

Hello Helge,

I looked at both commits [1] and pull requests [2].  I found no commits or pull 
requests that plausibly could explain the drop in code smells from March 26 to 
April 27 on the chart [3] you linked to.  No one had mentioned doing anything 
to reduce SonarQube code smells on the dev or users lists either.  I think you 
are correct that the configuration of SonarCloud's quality profiles must have 
changed, although I don't know if any other Daffodil maintainer made a change 
or if SonarCloud made a change itself.  

Mike and Steve, did you change anything in the SonarCloud configuration either? 

John

[1] https://github.com/apache/daffodil/commits/master
[2] https://github.com/apache/daffodil/pulls?q=is%3Apr+is%3Aclosed
[3] 
https://sonarcloud.io/project/activity?id=apache-daffodil_date=2020-04-24T17%3A05%3A46%2B

-Original Message-
From: Helge Pfeiffer  
Sent: Wednesday, June 16, 2021 5:38 PM
To: dev@daffodil.apache.org
Subject: EXT: SonarCloud Code Smell Reduction?

Dear Daffodil developers,

My name is Helge, I am a researcher at IT University of Copenhagen [1].
I am currently conducting a study on the impact of continuous code quality 
assessment tools (SonarQube) on defects.

I am writing to you -the Daffodil developers-, since I found that Daffodil uses 
SonarCloud for continuous code quality assessment, that it is the ASF project 
with the lowest amount of code smells, vulnerabilities and 'bugs' in 
SonarCloud, and that it is the project with the biggest reduction of code 
smells (drop of ca. 60% within around 2 months) [2].

However, I am wondering if the drastic reduction of code smells that are 
reported by SonarCloud is due to code changes that address these issues or if 
it is caused by configuration of the rules ("quality profile") that SonarCloud 
applies.

I believe the latter is the case. I can only find 5 commits that are related to 
SonarQube/-Cloud or any of the reported code smells, vulnerabilities, or bugs 
[3]. I identified these commits by searching for `[Ss]mell`, 
`[Vv]ulnerabilit[iy]`, `[Bb]ug`, and `[Ss]onar` in the commit history and the 
Jira issue tracker.
I cannot see that these 5 commits are addressing multiple hundreds of 
SonarCloud code smells. However, I can see many changes of the kind `Quality 
Profile:Changes in 'Sonar way'` especially in the beginning of SonarClouds 
project activity [4].
Another possibility is of course that I just do not find the commits that 
address the SonarCloud code smells.

Therefore, I would really appreciate your feedback to my question:

  1) Is the drastic reduction of code smells that SonarCloud reports for 
Daffodil due to configuration of SonarCloud's quality profiles?
1.1) If not, could you please point me to some of the commits that address 
code smells and that I fail to identify?



Thank you in advance for your feedback and consideration. I will share the 
results of my work with you as soon they are written down in a presentable 
format.


Best regards,
Helge



---

[1] https://www.itu.dk/people/ropf/ and 
https://www.researchgate.net/profile/Helge-Pfeiffer-2
[2] https://sonarcloud.io/project/activity?id=apache-daffodil
[3] Commits related to SonarQube/-Cloud, code smells, vulnerabilities, or bugs
  * 
https://github.com/apache/daffodil/commit/2426e7f8527c289937506178a0e65da421d999ea
  * 
https://github.com/apache/daffodil/commit/f3eee732f1f5535d0177877720c4fe9f39bc3327
  * 
https://github.com/apache/daffodil/commit/075ed018d786d332deddc5e20169939f95470fef
  * 
https://github.com/apache/daffodil/commit/8bcd8ef9440a890156915377bf55bf21047660dd
  * 
https://github.com/apache/daffodil/commit/b1d4c5412db985ecfdbb6fa6c860f8205991b902

[4] 
https://sonarcloud.io/project/activity?id=apache-daffodil_date=2020-04-24T17%3A05%3A46%2B