2

i have some R scripts, which only can run in sequential way, cannot be broken into chunks or any parallel library for R or any other language cannot be used.

Is there any way i can distribute the Sequential execution of code to multiple cores or may be multiple servers in network? to speed up execution ?

Farhan
  • 4,377
  • 12
  • 56
  • 87

1 Answers1

0

So let us assume you can run:

./my_script.R arg1

And you want to run the script on arg1..arg1000. Then you can use GNU Parallel:

parallel ./my_script.R {} ::: arg1 arg2 arg3 .. arg1000

This will start one my_script.R per CPU thread.

If you have multiple servers at your disposal that you can ssh to:

parallel -Sserver1 -Sserver2 ./my_script.R {} ::: arg1 arg2 arg3 .. arg1000

If this does not answer your question, please elaborate on your situation.

Ole Tange
  • 3,186