Each text can be on a new line or you can define separator (i.e. Moreover, shuf does all the work in memory. Enter data you want to shuffle and let our online randomizer do the job. That’s because the sort command must compute a hash key for every single line, which is a quite expensive operation, while the shuf command doesn’t have this calculation. The result shows that the sort -R approach is much (about 50 times) slower than the one with shuf. Let’s use the powerful awk command to prepend a random number on each line: $ awk 'BEGIN' big.txt \ denote i as the line number use idx j as the row that should go to the sample file. idx Get-Random -count controlsize -InputObject (0. Make sure to pipe through sort at the end. The sort command in the middle sorts the prepared line by the first field, which contains the random numbers we prepended. Create an index of random numbers to determine which rows should go to sample file. ![]() Maybe there is also an easy/simple awk command I do not know about that could handle this problem. I have heard of shuf but I do not know how to use it for two files. ![]() Finally, we’ll need to build our command like: cmd-to-prepend-random-number-on-each-line | sort -n -k 1 | cmd-to-remove-random-number-prefix-from-each-line What I have are two text files that I need to shuffle randomly, but I need the two files to be randomly shuffled the same way.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. Archives
June 2023
Categories |