Re: [Development] date/time adjust for (auto) testing
On Wednesday 03 June 2015 08:00:54 Curtis Mitch wrote: Sounds like a great thing to have if you can get it working. The QDateTime tests themselves would benefit from this quite a lot. Perhaps you've already seen, but it looks like Thiago attempted something similar with a8c74ddcf78604c9038ba2a2bea81e445e4b3c58: http://code.qt.io/cgit/qt/qtbase.git/tree/tests/auto/corelib/tools/qdatetime /tst_qdatetime.cpp#n2988 That's not the same. Timezones on Linux are a property of the application and can easily and trivially be changed. Changing the time is much more difficult. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] date/time adjust for (auto) testing
On Wednesday 03 June 2015 08:38:48 André Somers wrote: Hi, We have applications that use the current date and time at places spread around the code. For normal operations, that works very nicely. However, we find that for (auto) testing, it would be very convenient if we could trick the application into believing it is some other date/time, so that we can test if certain behaviours work the way we would like to automatically. Currently, these tests take a lot of time because we actually need to manually adjust the system date and time, do some stuff, then adjust again, etc. It would be really confortable if there was some control to set a date/time offset (so the time keeps running) or a fixed date/time to be returned from currentDate(), currentTime() or currentDateTime() respectively. I guess access to such a thing does not belong in the main Qt classes, but is really a testing tool, so perhaps it could find refuge in QtTest somewhere. Would a contribution adding such a thing stand any chance of being accepted, or would this be considered out of scope or even unwanted? Yes, I think so. The only gotcha is that the functions that would most benefit from the feature *are* the functions you mentioned. It would be so nice to properly test that we handle daylight savings transitions correctly. So what functions / functionality are you thinking of testing that would require this? One way to do that is to use ELF interception of the glibc gettimeofday() function. On Windows, interception is also possible, but requires a very ugly hack. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] date/time adjust for (auto) testing
Bo Thorsen schreef op 3-6-2015 om 11:44: Den 03-06-2015 kl. 08:38 skrev André Somers: We have applications that use the current date and time at places spread around the code. For normal operations, that works very nicely. However, we find that for (auto) testing, it would be very convenient if we could trick the application into believing it is some other date/time, so that we can test if certain behaviours work the way we would like to automatically. Currently, these tests take a lot of time because we actually need to manually adjust the system date and time, do some stuff, then adjust again, etc. It would be really confortable if there was some control to set a date/time offset (so the time keeps running) or a fixed date/time to be returned from currentDate(), currentTime() or currentDateTime() respectively. I guess access to such a thing does not belong in the main Qt classes, but is really a testing tool, so perhaps it could find refuge in QtTest somewhere. Would a contribution adding such a thing stand any chance of being accepted, or would this be considered out of scope or even unwanted? An alternative might be to hook the windows kernel API, but that may be much tricker to get right and may have unforseen consequences for the code injected by Squish doing the actual testing. To me it sounds like you need to create your own static current...() functions so you can control the output for unit tests instead. That might be an option, and I did considder that. We may end up doing that, but it would involve changing quite a number of places and worse, is a bit hard to make sure is used everywhere consistently in the future. Or even better: Modify the tests so you can adapt the compares to something that handles the current time and date. Date is simple enough (yes, it can fail if you run the test at exactly 23.59.59.9). For the time you just allow for a couple of seconds difference. That's much harder to do, as I already explained earlier. I *really* don't like the idea of creating a system to add offsets to the QDateTime::currentDateTime (or friends). Then we can't depend on the output being correct, which means any unit tests you write using it are useless. I expected the proposel to be contriversial, that's why I posted it just as an email in the first place. I think it would be useful especially _for_ unit tests. I'd also put the controls in the QtTest namespace, so that it is very obvious that this functionality is there to facilitate tests, not for other use. Note that there are many other methods there to trick Qt into acting as if events happened on the outside of the application, like simulating clicks and keyboard actions. Here, one would simulate the change of system time. If you really want to modify the output of current...() then you might consider preloading a library over Qt that does it for you. I am considering that option as well, but that would require me to hook into the windows kernel calls. Like I said, that approach is much trickier and have a bigger chance of having side effects on for instance Squish. André ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] date/time adjust for (auto) testing
-Original Message- From: development-bounces+mitch.curtis=theqtcompany@qt-project.org [mailto:development-bounces+mitch.curtis=theqtcompany@qt-project.org] On Behalf Of André Somers Sent: Wednesday, 3 June 2015 8:39 AM To: development@qt-project.org Subject: [Development] date/time adjust for (auto) testing Hi, We have applications that use the current date and time at places spread around the code. For normal operations, that works very nicely. However, we find that for (auto) testing, it would be very convenient if we could trick the application into believing it is some other date/time, so that we can test if certain behaviours work the way we would like to automatically. Currently, these tests take a lot of time because we actually need to manually adjust the system date and time, do some stuff, then adjust again, etc. It would be really confortable if there was some control to set a date/time offset (so the time keeps running) or a fixed date/time to be returned from currentDate(), currentTime() or currentDateTime() respectively. I guess access to such a thing does not belong in the main Qt classes, but is really a testing tool, so perhaps it could find refuge in QtTest somewhere. Would a contribution adding such a thing stand any chance of being accepted, or would this be considered out of scope or even unwanted? An alternative might be to hook the windows kernel API, but that may be much tricker to get right and may have unforseen consequences for the code injected by Squish doing the actual testing. André -- ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development Sounds like a great thing to have if you can get it working. The QDateTime tests themselves would benefit from this quite a lot. Perhaps you've already seen, but it looks like Thiago attempted something similar with a8c74ddcf78604c9038ba2a2bea81e445e4b3c58: http://code.qt.io/cgit/qt/qtbase.git/tree/tests/auto/corelib/tools/qdatetime/tst_qdatetime.cpp#n2988 ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] date/time adjust for (auto) testing
Den 03-06-2015 kl. 08:38 skrev André Somers: We have applications that use the current date and time at places spread around the code. For normal operations, that works very nicely. However, we find that for (auto) testing, it would be very convenient if we could trick the application into believing it is some other date/time, so that we can test if certain behaviours work the way we would like to automatically. Currently, these tests take a lot of time because we actually need to manually adjust the system date and time, do some stuff, then adjust again, etc. It would be really confortable if there was some control to set a date/time offset (so the time keeps running) or a fixed date/time to be returned from currentDate(), currentTime() or currentDateTime() respectively. I guess access to such a thing does not belong in the main Qt classes, but is really a testing tool, so perhaps it could find refuge in QtTest somewhere. Would a contribution adding such a thing stand any chance of being accepted, or would this be considered out of scope or even unwanted? An alternative might be to hook the windows kernel API, but that may be much tricker to get right and may have unforseen consequences for the code injected by Squish doing the actual testing. To me it sounds like you need to create your own static current...() functions so you can control the output for unit tests instead. Or even better: Modify the tests so you can adapt the compares to something that handles the current time and date. Date is simple enough (yes, it can fail if you run the test at exactly 23.59.59.9). For the time you just allow for a couple of seconds difference. I *really* don't like the idea of creating a system to add offsets to the QDateTime::currentDateTime (or friends). Then we can't depend on the output being correct, which means any unit tests you write using it are useless. If you really want to modify the output of current...() then you might consider preloading a library over Qt that does it for you. Bo Thorsen, Director, Viking Software. -- Viking Software Qt and C++ developers for hire http://www.vikingsoft.eu ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] date/time adjust for (auto) testing
* André Somers an...@familiesomers.nl [2015-06-03 08:38:48 +0200]: We have applications that use the current date and time at places spread around the code. For normal operations, that works very nicely. However, we find that for (auto) testing, it would be very convenient if we could trick the application into believing it is some other date/time, so that we can test if certain behaviours work the way we would like to automatically. Currently, these tests take a lot of time because we actually need to manually adjust the system date and time, do some stuff, then adjust again, etc. It would be really confortable if there was some control to set a date/time offset (so the time keeps running) or a fixed date/time to be returned from currentDate(), currentTime() or currentDateTime() respectively. I guess access to such a thing does not belong in the main Qt classes, but is really a testing tool, so perhaps it could find refuge in QtTest somewhere. Would a contribution adding such a thing stand any chance of being accepted, or would this be considered out of scope or even unwanted? FWIW there's libfaketime[1] which might be some inspiration - but that's Linux/OS X only. [1] https://github.com/wolfcw/libfaketime Florian -- http://www.the-compiler.org | m...@the-compiler.org (Mail/XMPP) GPG: 916E B0C8 FD55 A072 | http://the-compiler.org/pubkey.asc I love long mails! | http://email.is-not-s.ms/ pgpT7qUNSlx4m.pgp Description: PGP signature ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] date/time adjust for (auto) testing
Hi, I also had the same needs for testing my app, on different times of the day or year etc. But instead of patching Windows or some Qt code, I created a new virtual Windows VM in WMWare, and deployed my app into it. Switched off Synchronize guest time with host in the Options, and whenever I test, first a .bat file is run that sets the requested date/time in the Windows VM. This can you do in VirtualBox as well. Note that when you run on a fake date/time, https (certificates) and a lot of other internet stuff will fail, but for local app testing it's ok. Rgrds Henry On 2015-06-03 08:38, André Somers wrote: Hi, We have applications that use the current date and time at places spread around the code. For normal operations, that works very nicely. However, we find that for (auto) testing, it would be very convenient if we could trick the application into believing it is some other date/time, so that we can test if certain behaviours work the way we would like to automatically. Currently, these tests take a lot of time because we actually need to manually adjust the system date and time, do some stuff, then adjust again, etc. It would be really confortable if there was some control to set a date/time offset (so the time keeps running) or a fixed date/time to be returned from currentDate(), currentTime() or currentDateTime() respectively. I guess access to such a thing does not belong in the main Qt classes, but is really a testing tool, so perhaps it could find refuge in QtTest somewhere. Would a contribution adding such a thing stand any chance of being accepted, or would this be considered out of scope or even unwanted? An alternative might be to hook the windows kernel API, but that may be much tricker to get right and may have unforseen consequences for the code injected by Squish doing the actual testing. André ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] date/time adjust for (auto) testing
Curtis Mitch schreef op 3-6-2015 om 10:00: -Original Message- From: development-bounces+mitch.curtis=theqtcompany@qt-project.org [mailto:development-bounces+mitch.curtis=theqtcompany@qt-project.org] On Behalf Of André Somers Sent: Wednesday, 3 June 2015 8:39 AM To: development@qt-project.org Subject: [Development] date/time adjust for (auto) testing Hi, We have applications that use the current date and time at places spread around the code. For normal operations, that works very nicely. However, we find that for (auto) testing, it would be very convenient if we could trick the application into believing it is some other date/time, so that we can test if certain behaviours work the way we would like to automatically. Currently, these tests take a lot of time because we actually need to manually adjust the system date and time, do some stuff, then adjust again, etc. It would be really confortable if there was some control to set a date/time offset (so the time keeps running) or a fixed date/time to be returned from currentDate(), currentTime() or currentDateTime() respectively. I guess access to such a thing does not belong in the main Qt classes, but is really a testing tool, so perhaps it could find refuge in QtTest somewhere. Would a contribution adding such a thing stand any chance of being accepted, or would this be considered out of scope or even unwanted? An alternative might be to hook the windows kernel API, but that may be much tricker to get right and may have unforseen consequences for the code injected by Squish doing the actual testing. André -- ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development Sounds like a great thing to have if you can get it working. The QDateTime tests themselves would benefit from this quite a lot. Perhaps you've already seen, but it looks like Thiago attempted something similar with a8c74ddcf78604c9038ba2a2bea81e445e4b3c58: http://code.qt.io/cgit/qt/qtbase.git/tree/tests/auto/corelib/tools/qdatetime/tst_qdatetime.cpp#n2988 I did not see that yet. What I have in mind is not actually changing the system date/time/timezone, but changing what Qt returns for it to ease (regressino) testing. A concrete use case: We have a database with entries on a specific dates. We also have an export operation that allows you to specify a time period to export, with some default ranges defined: today, this week, this month. Obviously, that uses the current date and time. But that hinders testing, as an automatic test would preferably use a pre-made database that we just prepare once and then just run the test on. Running the test next week will yield different results from running the test this week, and that is hard to evaluate. On the other hand, creating a new database for the test is tricky, as records get their time stamp automatically from, again, the current date and time. So creating a new record that appears one month and a day old is just as tricky. We'd like to avoid creating special paths in the actual application code as much as we can, because that increases maintenance costs and diminishes the value of the actual test: we'd be testing a special code path no actual user will ever use. If we were able to manipulate the 'current' time and date returned by Qt's date/time classes, testing this would be easy. We'd just have the test script set a pre-defined 'current' date, and be able to auto-test everything against a pre-defined database. I would add something to the actual code of the current functions to adjust the result when such an adjustment is set. That would incur a small overhead in the actual function, even if not enabled I think. André ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
[Development] date/time adjust for (auto) testing
Hi, We have applications that use the current date and time at places spread around the code. For normal operations, that works very nicely. However, we find that for (auto) testing, it would be very convenient if we could trick the application into believing it is some other date/time, so that we can test if certain behaviours work the way we would like to automatically. Currently, these tests take a lot of time because we actually need to manually adjust the system date and time, do some stuff, then adjust again, etc. It would be really confortable if there was some control to set a date/time offset (so the time keeps running) or a fixed date/time to be returned from currentDate(), currentTime() or currentDateTime() respectively. I guess access to such a thing does not belong in the main Qt classes, but is really a testing tool, so perhaps it could find refuge in QtTest somewhere. Would a contribution adding such a thing stand any chance of being accepted, or would this be considered out of scope or even unwanted? An alternative might be to hook the windows kernel API, but that may be much tricker to get right and may have unforseen consequences for the code injected by Squish doing the actual testing. André -- ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] date/time adjust for (auto) testing
Den 03-06-2015 kl. 12:26 skrev André Somers: If you really want to modify the output of current...() then you might consider preloading a library over Qt that does it for you. I am considering that option as well, but that would require me to hook into the windows kernel calls. Like I said, that approach is much trickier and have a bigger chance of having side effects on for instance Squish. No, I meant preloading over the QDateTime::currentDateTime function. You can load a dll that replaces specific functions in another dll, which means you can replace any Qt method you want to. Or at least that's possible in theory. I've never actually done it on windows. I can perfectly understand you don't want to mess with windows kernel hooks. But doing dll injection on Qt itself might be okay for running your unit tests. On linux you can do this with LD_PRELOAD as well. Bo Thorsen, Director, Viking Software. -- Viking Software Qt and C++ developers for hire http://www.vikingsoft.eu ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
