Re: [DISCUSS] Drop Java 8?

[Jialin Qiao]

Hi,

Drop java 8 will bring some CVE fixed, but also reduce a common option
for users,  for this, I will -0.

Maintaining two branch will surely resolve the issue, but this will
bring much additional work... For this, I will -1

Although we put 0.13 and 1.x version in our download page, we actually
do not maintain the 0.x branch and there is on plan to release a new
0.x version.

So, as there is a lot to do with 1.x this year (the table schema, data
subscription, tsfile consensus), I prefer not to do the drop this
year.

Jialin Qiao

Christofer Dutz  于2024年5月17日周五 14:43写道：
>
> So we have real numbers?
>
> I also haven't used an oracle jre for quite some time for these reasons.
>
> Would a double branch strategy work? That the project maintains a 1.x branch 
> with Javax and Java 8 and a 2.0 branch with Jakarta and Java 11?
>
> Sort of like we had the 0.x and 1.x branches around?
>
> Chris
>
>
> Gesendet von Outlook für Android
> 
> From: Trevor Hart 
> Sent: Friday, May 17, 2024 4:01:14 AM
> To: dev 
> Subject: Re: [DISCUSS] Drop Java 8?
>
> I think a lot of organisations are sticking to Java 8 because of the change 
> to Oracle license that was introduced in Java 11.
>
>
>
> If you use Oracle 11 JRE you need to pay Oracle for a license.
>
>
>
> This was why Open JDK came about.
>
>
>
> Personally I use IotDB with Open JDK 11 (Eclipse Temurin) which does not 
> require a license.
>
>
>
> Thanks
>
> Trevor Hart
>
>
>
>
>
>
>
>
>  On Fri, 17 May 2024 13:30:15 +1200 Yuan Tian  
> wrote ---
>
>
>
> Hi Chris,
>
> It seems that a lot of people still use jdk1.8 in their product environment.
>
> Best regards,
> -
> Yuan Tian
>
> On Thu, May 16, 2024 at 8:10 PM Christofer Dutz 
> 
> wrote:
>
> > Hi all,
> >
> > starting this new thread as I am not sure if others are reading the
> > Jakarta migration thread.
> >
> > I would like to propose planning on dropping Java 8 support.
> >
> > I wouldn’t immediately do that, and I would also propose to do a major
> > version update (Switching to 2.0.0)
> >
> > We could still maintain a 1.x branch for those people not able to update.
> >
> > The main reason is that we are currently blocking ourselves from updating
> > many major plugins and dependencies.
> > I noticed that when updating to the Jakarta namespace. Here there is no
> > Netty version available that supports Jakarta and supports Java 8.
> >
> > Other libraries where we are not able to update without giving up on Java
> > 8:
> >
> >   *   Airlift-Units (Stuck at 1.7 current 1.10)
> >   *   Airlift
> >   *   Antlr (Stuck at 4.9.3 current 4.13.1)
> >   *   Caffeine (Stuck at 2.9.3 current 3.1.8)
> >   *   Logback (Stuck at 1.3.14 current 1.5.6)
> >   *   Mockito (Stuck at 2.23.4 current 5.12.0)
> >   *   Thrift (Stuck at 0.17.0 current 0.20.0)
> >
> >
> >
> >   *   Spotless Plugin (We’ve got a workaround for Java 8)
> >
> >
> > In my branch where I refactored the javax namespace to Jakarta after
> > updating dependencies I was able to remove all exclusions of the
> > BanVulnerableDependencies check.
> >
> > Also does dropping Java 8 and the Jakarta migration allow embedding IoTDB
> > in recent Spring versions.
> >
> >
> > So … what do you think?
> >
> >
> > Chris
> >

AW: Possible fix for randomly failing Jenkins builds

[Christofer Dutz]

Done:
https://github.com/apache/iotdb/pull/12542
Waiting for approval.

Chris

Von: Jialin Qiao 
Datum: Freitag, 17. Mai 2024 um 09:15
An: dev@iotdb.apache.org 
Betreff: Re: Possible fix for randomly failing Jenkins builds
+1 for importing the solutions in PLC4X


Jialin Qiao

Christofer Dutz  于2024年5月14日周二 23:32写道：
>
> Hi all,
>
> I know that in IoTDB as well as PLC4X we were having issues of regularly but 
> randomly failing Jenkins builds.
>
> Usually, these problems were because of timeouts when deploying to Nexus.
>
> In PLC4X I have now tried something that seems to be doing the job … possibly 
> this could also help us here (I mean … the biggest part of the Jenkins setup 
> is borrowed from PLC4X anyway ;-) )
>
> My solution was to wrap the execution of the deploy step in a loop on bash 
> level:
>
> https://github.com/apache/plc4x/blob/develop/Jenkinsfile#L167
>
> sh 'until ./mvnw -f jenkins.pom -X -P deploy-snapshots 
> wagon:upload || (( count++ >= 5 )); do echo "Retrying to deploy"; done'
>
> So, it would try up to 5 times, if deployment fails.
>
> I think in case of deployment the full build is done and was successful. The 
> only reason for failures in this step usually is problems with Nexus and a 
> stupid retry should help.
>
> What do you think? Should we try the same change?
>
>
> Chris

Re: Possible fix for randomly failing Jenkins builds

[Jialin Qiao]

+1 for importing the solutions in PLC4X


Jialin Qiao

Christofer Dutz  于2024年5月14日周二 23:32写道：
>
> Hi all,
>
> I know that in IoTDB as well as PLC4X we were having issues of regularly but 
> randomly failing Jenkins builds.
>
> Usually, these problems were because of timeouts when deploying to Nexus.
>
> In PLC4X I have now tried something that seems to be doing the job … possibly 
> this could also help us here (I mean … the biggest part of the Jenkins setup 
> is borrowed from PLC4X anyway ;-) )
>
> My solution was to wrap the execution of the deploy step in a loop on bash 
> level:
>
> https://github.com/apache/plc4x/blob/develop/Jenkinsfile#L167
>
> sh 'until ./mvnw -f jenkins.pom -X -P deploy-snapshots 
> wagon:upload || (( count++ >= 5 )); do echo "Retrying to deploy"; done'
>
> So, it would try up to 5 times, if deployment fails.
>
> I think in case of deployment the full build is done and was successful. The 
> only reason for failures in this step usually is problems with Nexus and a 
> stupid retry should help.
>
> What do you think? Should we try the same change?
>
>
> Chris

Re: Handling Duplicate Timestamps

[Jialin Qiao]

Hi Trevor,

Will different values of the same timestamp be the same？

1. Same
Time, Value
1, 1
1, 1
1, 1

2. Different
Time, Value
1, 1
1, 2
1, 1


Jialin Qiao

Trevor Hart  于2024年5月14日周二 11:20写道：
>
> Thank you! I will implment some work around for now.
>
>
> I would appreciate some consideration for this option in the future.
>
>
> Thanks
>
> Trevor Hart
>
> Ope Limited
>
> w: http://www.ope.nz/
>
> m: +64212728039
>
>
>
>
>
>
>
>
>  On Tue, 14 May 2024 15:17:47 +1200 Xiangdong Huang  
> wrote ---
>
>
>
> > 1. Checking before insert if the timestamp already exists and remedy on the 
> > client before resend
> > 2. Moving to Nanosecond and introducing some insignificant time value to 
> > keep timestamp values unique.
> Yes these maybe the best solutions for a specific application.
>
>
> Analysis for IoTDB:
> - Rejecting the write when receiving an existing timestamp in IoTDB is
> time-costly (IoTDB needs to check historical data). I think we will do
> not check it until we find a low-latency method.
> - Allowing multiple value versions for a timestamp may introduce a
> chain reaction and there may be a lot of codes that should be
> modified, which is a huge work.
>
> There is a new idea (but I have no time to implement it...)
> - Add a parameter in IoTDB: replace_strategy: first, last, avg etc...
> - when an existing timestamp arrives, IoTDB accepts it
> - when IoTDB runs LSM to merge data and meets multiple values for a
> timestamp, then handles it according to the replace_startegy.
>
> The solution may also introduce some work to do... and we need to
> think carefully the impact to the query process.
> Need to survey whether this is a common requirement.
>
> Best,
> ---
> Xiangdong Huang
>
> Trevor Hart  于2024年5月14日周二 09:55写道：
> >
> > Hello Yuan
> >
> >
> >
> > Correct, the first timestamp and values should be retained.
> >
> >
> >
> > I realise this is does not align with the current design. I was just asking 
> > whether there was an existing option to operate to block duplicates.
> >
> >
> >
> > In a normal RDBMS if you try to insert with a duplicate the insert will 
> > fail with a PK violation. It would be great in some circumstances if IotDB 
> > at least had the option to fail this way.
> >
> >
> >
> > I am considering some options such as;
> >
> >
> >
> > 1. Checking before insert if the timestamp already exists and remedy on the 
> > client before resend
> >
> > 2. Moving to Nanosecond and introducing some insignificant time value to 
> > keep timestamp values unique.
> >
> >
> >
> > I have already done something similar to #2 with storing IIS web log files 
> > as they are recorded in seconds and not milliseconds.
> >
> >
> >
> > Thanks
> >
> > Trevor Hart
> >
> >
> >
> >
> >  On Tue, 14 May 2024 13:29:02 +1200 Yuan Tian 
> >  wrote ---
> >
> >
> >
> > Hi Trevor,
> >
> > By "rejects duplicates", you mean you want to keep the first duplicate
> > timestamp and its corresponding values?(because the following duplicated
> > ones will be rejected)
> >
> > Best regards,
> > 
> > Yuan Tian
> >
> > On Mon, May 13, 2024 at 6:24 PM Trevor Hart  
> > wrote:
> >
> > >
> > >
> > >
> > >
> > > Correct. I’m not disputing that. What I’m asking is that it
> > > would be good to have a configuration that either allows overwrites or
> > > rejects duplicates.My scenario is request log data from a server (the
> > > device). As it may be processing multiple requests at once there is a
> > > chance that there could be colliding time stamps.As it stands now I would
> > > need to check if the timestamp exists before inserting the data. Which
> > > obviously affects throughput. Thanks Trevor Hart On Fri, 10 May
> > > 2024 00:33:40 +1200  Jialin Qiao 
> > > wrote  Hi,
> > > In IoT or IIoT scenarios, we thought each data point represent a metric of
> > > a timestamp.In which case you need to store duplicated values?  Take this
> > > for an example: Time, root.sg1.car1.speed 1, 1 1, 2  Could a car has
> > > different speed at time 1?   Jialin Qiao  Yuan Tian <
> > > mailto:mailto:jackietie...@gmail.com> 于2024年5月9日周四 18:51写道： > > Hi 
> > > Trevor, > > Now we
> > > will override the duplicate timestamp with a newer one. There is > nothing
> > > we can do about it now. > > Best regards, > --- > Yuan 
> > > Tian
> > > > > On Wed, May 8, 2024 at 5:31 PM Trevor Hart 
> > > > >  wrote: > >
> > > > Hello > > > > > > > > I’m aware that when inserting a duplicate 
> > > > timestamp
> > > the values will be > > overwritten. This will obviously result in data
> > > loss. > > > > > > > > Is there a config/setting to reject or throw an 
> > > error
> > > on duplicate > > inserts? Although highly unlikely I would prefer to be
> > > alerted to the > > situation rather than lose data. > > > > > 

Re: [DISCUSS] Drop Java 8?

[Christofer Dutz]

So we have real numbers?

I also haven't used an oracle jre for quite some time for these reasons.

Would a double branch strategy work? That the project maintains a 1.x branch 
with Javax and Java 8 and a 2.0 branch with Jakarta and Java 11?

Sort of like we had the 0.x and 1.x branches around?

Chris


Gesendet von Outlook für Android

From: Trevor Hart 
Sent: Friday, May 17, 2024 4:01:14 AM
To: dev 
Subject: Re: [DISCUSS] Drop Java 8?

I think a lot of organisations are sticking to Java 8 because of the change to 
Oracle license that was introduced in Java 11.



If you use Oracle 11 JRE you need to pay Oracle for a license.



This was why Open JDK came about.



Personally I use IotDB with Open JDK 11 (Eclipse Temurin) which does not 
require a license.



Thanks

Trevor Hart








 On Fri, 17 May 2024 13:30:15 +1200 Yuan Tian  
wrote ---



Hi Chris,

It seems that a lot of people still use jdk1.8 in their product environment.

Best regards,
-
Yuan Tian

On Thu, May 16, 2024 at 8:10 PM Christofer Dutz 

wrote:

> Hi all,
>
> starting this new thread as I am not sure if others are reading the
> Jakarta migration thread.
>
> I would like to propose planning on dropping Java 8 support.
>
> I wouldn’t immediately do that, and I would also propose to do a major
> version update (Switching to 2.0.0)
>
> We could still maintain a 1.x branch for those people not able to update.
>
> The main reason is that we are currently blocking ourselves from updating
> many major plugins and dependencies.
> I noticed that when updating to the Jakarta namespace. Here there is no
> Netty version available that supports Jakarta and supports Java 8.
>
> Other libraries where we are not able to update without giving up on Java
> 8:
>
>   *   Airlift-Units (Stuck at 1.7 current 1.10)
>   *   Airlift
>   *   Antlr (Stuck at 4.9.3 current 4.13.1)
>   *   Caffeine (Stuck at 2.9.3 current 3.1.8)
>   *   Logback (Stuck at 1.3.14 current 1.5.6)
>   *   Mockito (Stuck at 2.23.4 current 5.12.0)
>   *   Thrift (Stuck at 0.17.0 current 0.20.0)
>
>
>
>   *   Spotless Plugin (We’ve got a workaround for Java 8)
>
>
> In my branch where I refactored the javax namespace to Jakarta after
> updating dependencies I was able to remove all exclusions of the
> BanVulnerableDependencies check.
>
> Also does dropping Java 8 and the Jakarta migration allow embedding IoTDB
> in recent Spring versions.
>
>
> So … what do you think?
>
>
> Chris
>