Re: Generate Digest MD5 - Max Input File Size?

[Keisuke Miyako]

Hash.4dbase is a v15 sample database (plugin preinstalled),
v11/Hash.4dbase is a v11 sample database (plugin preinstalled).

the v15 plugin can be used in all versions above v14, including Mac 64 bits.
the v11 plugin can be used in all versions above v11, excluding Mac 64 bits.

you can go inside the appropriate ".4dbase" package/folder
and find the ".bundle" plugin inside the Plugins folder.

> 2016/10/22 12:02、David Belote  のメール：
>  I tried copying the hash folder into the Plugins folder that I created, but 
> when I run the app, I can’t seem to make it work so I’m thinking that is not 
> right.  Any help you can provide about how to install 4d-plugin-hash on my 
> Mac would be greatly appreciated.



宮古 啓介
セールス・エンジニア

株式会社フォーディー・ジャパン
〒150-0043
東京都渋谷区道玄坂1-10-2 渋谷THビル6F
Tel: 03-6427-8441
Fax: 03-6427-8449

keisuke.miy...@4d.com
www.4D.com/JP

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Generate Digest MD5 - Max Input File Size?

[David Belote]

Wow!  This thread has yielded a wealth of information about hashes & checksums 
and the issues surrounding generating and using them.  David Adams’s Hash Tool 
Component PDF finally gave me a good explanation about what a Hash is and 
provides a good resource.  I’m still a little bit foggy about the difference 
between what a Hash is vs. what a Checksum is.  Are they really different?  
Since the 4D Language itself doesn’t use the concept of hashes (I know they are 
used in other programming languages extensively), I simply don’t know much 
about them.

I downloaded Keisuke’s Hash plugin and will be testing it for use in my app 
during the next few days to generate MD5 checksums on files.

David Belote
Colorado Springs  

On 10/20/16, 8:25 PM, "4D_Tech on behalf of Keisuke Miyako" 
<4d_tech-boun...@lists.4d.com on behalf of keisuke.miy...@4d.com> wrote:

it never occurred to me that PHP hashing could take so long...

here I've just posted a plugin to generate the hash of a file:

https://github.com/miyako/4d-plugin-hash

for what it's worth, you can pass a callback method too...

> 2016/10/20 23:17、Cannon Smith  のメール：
>
> I created a feature request a few years ago 
(http://forums.4d.fr/Post/EN/13706008/1/13706009#13706009) because I was in the 
same situation you are and turned to php as well. Only I was doing a SHA1 
instead of MD5. Anyway, the thing I was running into wasn’t a file size limit 
but a time limit. PHP calls error out after 30 seconds which wasn’t enough time 
to compute the SHA1 on the size of files I was working with.
>
> I just mention it in case this is what is happening to you as well.



宮古 啓介
セールス・エンジニア

株式会社フォーディー・ジャパン
〒150-0043
東京都渋谷区道玄坂1-10-2 渋谷THビル6F
Tel: 03-6427-8441
Fax: 03-6427-8449

keisuke.miy...@4d.com
www.4D.com/JP

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**


**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Generate Digest MD5 - Max Input File Size?

[Keisuke Miyako]

the "Generate digest" function is mainly used for hashing passwords or 
relatively small documents.
http://livedoc.4d.com/4D-Language-Reference-15-R5/Tools/Generate-digest.301-2937745.en.html

still, the code

$error:=_o_AP Get file MD5 digest($path;$hash)

works in v15 (32 bits)

http://livedoc.4d.com/4D-Pack-15-R5.1550/4D-Pack-Utilities/o-AP-Get-file-MD5-digest.301-2965838.en.html

> 2016/10/20 13:23、David Belote  のメール：
>
> The code that was failing on the 687.6 MB file with an out of memory error 
> follows.  In order to use the Generate digest command on a file, you have to 
> convert the file to a BLOB first.  If I would have written it, I would have 
> just read in the file directly and computing the checksum as it streamed in, 
> but for some reason the method’s author chose to allow only BLOB or Text as 
> input.



宮古 啓介
セールス・エンジニア

株式会社フォーディー・ジャパン
〒150-0043
東京都渋谷区道玄坂1-10-2 渋谷THビル6F
Tel: 03-6427-8441
Fax: 03-6427-8449

keisuke.miy...@4d.com
www.4D.com/JP

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Generate Digest MD5 - Max Input File Size?

[Cannon Smith]

Hi David,

I created a feature request a few years ago 
(http://forums.4d.fr/Post/EN/13706008/1/13706009#13706009) because I was in the 
same situation you are and turned to php as well. Only I was doing a SHA1 
instead of MD5. Anyway, the thing I was running into wasn’t a file size limit 
but a time limit. PHP calls error out after 30 seconds which wasn’t enough time 
to compute the SHA1 on the size of files I was working with.

I just mention it in case this is what is happening to you as well.

--
Cannon Smith
Synergy Farm Solutions Inc.
Hill Spring, AB Canada
403-626-3236




> On Oct 19, 2016, at 10:05 PM, David Belote  wrote:
> 
> I solved my MD5 large file issue using PHP, but I would still like to know 
> the input file size limits of the Generate digest command.
> 
> Using 4D’s PHP Execute, I generated an MD5 checksum on the same 687.7 MB 
> movie file that the 4D Generate digest failed on due to not enough memory.  
> 
> Following is the code that I used (taken from the example in the PHP modules 
> support in the 4D PHP documentation.
> 
> C_TEXT($md5Result)
> $ok:=PHP Execute("";"md5";$md5Result;$AbsolutePath)
> 

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Generate Digest MD5 - Max Input File Size?

[David Belote]

The code that was failing on the 687.6 MB file with an out of memory error 
follows.  In order to use the Generate digest command on a file, you have to 
convert the file to a BLOB first.  If I would have written it, I would have 
just read in the file directly and computing the checksum as it streamed in, 
but for some reason the method’s author chose to allow only BLOB or Text as 
input.

C_POINTER($ptr_pathFilename;$1)
$ptr_pathFilename:=$1

C_BLOB($blobbedDocument)
DOCUMENT TO BLOB($ptr_pathFilename->;$blobbedDocument)

$0:=Generate digest($blobbedDocument;MD5 digest)

David Belote
Colorado Springs

On 10/19/16, 9:49 AM, "4D_Tech on behalf of Timothy Penner" 
<4d_tech-boun...@lists.4d.com on behalf of tpen...@4d.com> wrote:

The docs say that Generate Digest accepts either a BLOB or a TEXT variable:
http://livedoc.4d.com/4Dv15-R5/help/Command/en/page1147.html

The docs also say that both Blobs and Text variables are limited to 2 GB - 
here are the relevant excerpts:


BLOB http://livedoc.4d.com/4Dv15-R5/help/Title/en/page262.html#12745 

A BLOB field or variable is a series of bytes (from 0 to 2 GB in length) 
that you can address individually or by using the BLOB Commands. There is no 
expression of type BLOB.
Note: In the 4D Language Reference manual, BLOB parameters in command 
descriptions are denoted as BLOB.


String http://livedoc.4d.com/4Dv16-R2/help/Title/en/page262.html#1211073 

String is a generic term that stands for:
* Alphanumeric fields or variables: an Alphanumeric field may contain from 
0 to 255 characters (limit set when field is defined).
* Text fields or variables: a Text field, variable, or expression may 
contain from 0 to 2 GB of text..
* Any String or Text expression
There is no difference between a string or text variable.
You can assign a string to a text field and vice-versa; 4D does the 
conversion, truncating if necessary. You can mix string and text in an 
expression.
Note: In the 4D Language Reference manual, both string and text parameters 
in command descriptions are denoted as String, except when marked otherwise.


-Tim

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**


**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

RE: Generate Digest MD5 - Max Input File Size?

[Timothy Penner]

The docs say that Generate Digest accepts either a BLOB or a TEXT variable:
http://livedoc.4d.com/4Dv15-R5/help/Command/en/page1147.html

The docs also say that both Blobs and Text variables are limited to 2 GB - here 
are the relevant excerpts:


BLOB http://livedoc.4d.com/4Dv15-R5/help/Title/en/page262.html#12745 

A BLOB field or variable is a series of bytes (from 0 to 2 GB in length) that 
you can address individually or by using the BLOB Commands. There is no 
expression of type BLOB.
Note: In the 4D Language Reference manual, BLOB parameters in command 
descriptions are denoted as BLOB.


String http://livedoc.4d.com/4Dv16-R2/help/Title/en/page262.html#1211073 

String is a generic term that stands for:
* Alphanumeric fields or variables: an Alphanumeric field may contain from 0 to 
255 characters (limit set when field is defined).
* Text fields or variables: a Text field, variable, or expression may contain 
from 0 to 2 GB of text..
* Any String or Text expression
There is no difference between a string or text variable.
You can assign a string to a text field and vice-versa; 4D does the conversion, 
truncating if necessary. You can mix string and text in an expression.
Note: In the 4D Language Reference manual, both string and text parameters in 
command descriptions are denoted as String, except when marked otherwise.


-Tim

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Generate Digest MD5 - Max Input File Size?

[Charlie Vass]

Depends on which version of 4D you’re talking about. There are now 64 bit 
versions for Mac and Win.

Charlie

> On Oct 19, 2016, at 10:18 AM, Jeffrey Kain  wrote:
> 
> 4D is a 32-bit application, so it's probably somewhere shy of 2GB since you 
> need to load the document into a blob in memory. How big are your files?
> 
>> On Oct 19, 2016, at 10:08 AM, David Belote  wrote:
>> 
>> What is the maximum file size that the “Generate digest” command can process 
>> when computing an MD5 checksum?
> 
> **
> 4D Internet Users Group (4D iNUG)
> FAQ:  http://lists.4d.com/faqnug.html
> Archive:  http://lists.4d.com/archives.html
> Options: http://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
> **

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: Generate Digest MD5 - Max Input File Size?

[Jeffrey Kain]

4D is a 32-bit application, so it's probably somewhere shy of 2GB since you 
need to load the document into a blob in memory. How big are your files?

> On Oct 19, 2016, at 10:08 AM, David Belote  wrote:
> 
> What is the maximum file size that the “Generate digest” command can process 
> when computing an MD5 checksum?

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**