Repository navigation
Reuse of createHash initializations #25857
Description
Reuse is a fundamental issue...
const crypto = require('crypto')
const H = crypto.createHash('md5') // NEED REUSE THIS INITIALIZATION
console.log( H.update('some data to hash').digest('hex') ) // first ok...
console.log( H.update('some2').digest('hex') ) // ugly "Error: Digest already called"!The problem that I am trying to solve: to reuse initialization.
The desired behavior: no error.
... No alternative solutions, but please, show me if there are some elegant one.
Activity
[-]Reuse of hash initialization[/-][+]Reuse of createHash initializations[/+]sam-github commentedon Jan 31, 2019
Could you describe what it is you need, and why?
Your example isn't helpful. I don't know if you want openssl style digest reuse, where after _final() the only call that can be made is an init, after which new data can be digested, or whether you want to continue on hashing new data as if the digest had not been extracted.
The latter would require calling EVP_MD_CTX_copy() every time before getting the digest value, so would be prohibitively expensive. It would be possible to expose a
.copy()method on digests, if that's what you want, and you have a motivating use-case.ppKrauss commentedon Jan 31, 2019
Hi @sam-github ,
any loop needs reuse:
But the only valid (no error) syntax option is:
Please, show an example of reuse using the cited
.copy().Important: there are many
crypto.getHashes()algorithms... This question is about only one,sha256, there are 2,493 pageviws of users that did not understand "Digest already called" error... This error not make sense: because reuse make sense.sam-github commentedon Feb 1, 2019
This still does not answer my question, you provide code, but no explanation of how you want the code to work.
Specifically, will iteration 2 output the hash of "José Silva", or of "Ana SantosJosé Silva"?
ppKrauss commentedon Feb 1, 2019
@sam-github sorry, I edited with complete example... But really not understand what you show as solution.
sam-github commentedon Feb 1, 2019
It appears you want iteration 2 to output the hash of "José Silva", in other words a call to
.digest()will reset the internal state back to the initial just-constructed state, is this correct?I offered no solution. Until now, I wasn't even sure what the problem was.
I'll try to find some time to benchmark initialization to see what kind of performance gains are possible.
Providing your own benchmarks might help motivate someone to pick this up, or you could PR the change. It seems reasonable to me.
bnoordhuis commentedon Feb 2, 2019
It would be kind of odd (in a 'no parity' sense) to have that work for hashes but not e.g. HMACs.
(You could theoretically make it work for HMACs by retaining the secret but I don't think we should.)
sam-github commentedon Feb 5, 2019
Differing from HMAC would be a definite downside.
I notice that the openssl APIs have the ability to reuse the HASH context. I assume this is for use-cases where a HASH with some params is created, then used over and over to digest large numbers of high-throughput blocks/records/whatever, and avoid having to do a digest-init for each one.
That OpenSSL does it suggests it might be useful, but whether it is relevant to any node users, or even noticable if it was done is an open question. I keep hoping someone will come up with a use-case backed by benchmarks showing that this would have a significant impact for them, but I haven't seen it yet.
tniessen commentedon Feb 22, 2019
I did a quick'n'dirty benchmark to see how much time
createHash()needs in the following snippet (datais a buffer):datacreateHash('sha256')No surprises so far, the initialization time only matters for small inputs, and probably even less when dealing with strings instead of
Buffers. Three questions arise:ppKrauss commentedon Feb 22, 2019
Thanks @tniessen, perfect (!), now will be possible do finish this issue... after some discussion.
The main justification is not performance, but something like "syntax usability enhancement", as I commented before:
"... this question is about only one of many, the
sha256, there are 2,650 pageviws of users that did not understand 'Digest already called' error... This error not make sense: because reuse make sense".About performence: ok, the gain is 0 or, sometimes, greater tham zero. Is good, any gain will be a sum, a better justification, little better cost-benefit.
tniessen commentedon Feb 24, 2019
@ppKrauss You have expressed your personal opinion quite strongly, both here and on StackOverflow. Please keep the discussion productive with constructive arguments instead of repeating that things are "ugly" and that "reuse makes sense".
If the only benefit is "syntax usability enhancement", then consider the downsides:
Hashobjects in asynchronous code as soon as we allow reuse. I assume very few applications need to compute many hashes of small input values in a synchronous way.AshDee commentedon Apr 25, 2019
Regarding this thread.
In my code, I try to create hashes by reading different files by using createReadStream. But It is only possible to read one file. If I try to read the second file I get the error Digest already called.
From the documents, I came to know that digest can be used only once.
So, is there a workaround to read each file and generate corresponding hashes for each file?
As @sam-github had mentioned in his previous comment-
I would like to have something like this
tniessen commentedon Apr 26, 2019
@AshDee I don't understand, why don't you just create a new instance using
createHashfor each file? The additional overhead is minimal compared to IO delays.AshDee commentedon Apr 26, 2019
Thank You @tniessen .I'll try it out...
bnoordhuis commentedon Oct 9, 2019
I'm closing this out in favor of #29903.
hippyaki commentedon Mar 6, 2022
@ppKrauss Love you mate. Awesome solution with that. Works GREAT for me.