Ask HN: Is there a market for batched distributed Monte Carlo computation?

7 points

14 years ago

I've been working on an extension to the bitcoin protocol called NooShare in which participants can buy the right to specify a Monte Carlo calculation to be run as part of the blockchain's proof of work algorithm [1]. I'm pretty excited about this, because in principle it could be useful for a variety of optimization and inference problems, and even if it only got to be as big as litecoin [2] is today it would be a substantial computational resource. But I am also worried that it forces a model of computation which many potential users will find alien and difficult to program to [3]. Andrea Arcangeli's CPUShare was a similar marketplace for computational resources, with a similar computational model. CPUShare died, and one of the main frictions he faced in selling his service was that people wanted to just run their existing code, which relied on languages his security model couldn't handle, file system storage, real-time network communication, etc. [4]

I think building distributed computation into a blockchain the way I'm proposing with NooShare potentially solves a chicken-and-egg problem CPUShare probably had. That is, because all transactions were spot trades denominated in conventional currency, no one was interested in plugging their computers into the CPUShare network to sell time because no one else was buying time, and no one was buying time because the resources on sale were so small. Hopefully there will be a contingent of early NooShare miners who are excited about winning currency which will grow in value as the computational resources committed to the network grow, just like the early bitcoin adopters.

But, I am really concerned about the potential level of demand for NooShare's computational resources. Does it sound to people here like there would be a market for this? Any suggestions for how I could explore this question without implementing NooShare in its entirety? Something along the lines of "build an MVP involving just the computational framework you intend to use, and see whether people buy it," except that then I'm back in the same situation as CPUShare, with no substantial computational resources to motivate its adoption.

  Notes
  -----

  [1] http://tinyurl.com/nooshare
  
      (Google doc preview of pdf.  Click on File->Download Original in
      the top LH corner to view it locally.  Incidentally, I welcome
      feedback on any aspect of this paper and its ideas.  I'm asking
      here about marketability mainly because I think it's HN's
      wheelhouse, but I'd also love for people to pound on the
      economics, cryptography and security features of the design.)

  [2] http://litecoin.org/
  
  [3] Difficult aspects of the computational model: Batched calculations
      which will run as much as several days later; almost embarassingly
      parallel, with extremely limited communication restricted to a
      tiny number of processes run in one current block reporting their
      results to processes being run in subsequent blocks.  It would
      suffice to find the sweet spots in the parameter space of a
      Markov-Chain Monte Carlo algorithm, but I haven't thought of a way
      to generalize it beyond that.

  [4] http://tinyurl.com/7dtt536
  
      (archive.org copy of CPUShare blog post.)

2 comments

Notes ----- [1] http://tinyurl.com/nooshare (Google doc preview of pdf. Click on File->Download Original in the top LH corner to view it locally. Incidentally, I welcome feedback on any aspect of this paper and its ideas. I'm asking here about marketability mainly because I think it's HN's wheelhouse, but I'd also love for people to pound on the economics, cryptography and security features of the design.) [2] http://litecoin.org/ [3] Difficult aspects of the computational model: Batched calculations which will run as much as several days later; almost embarassingly parallel, with extremely limited communication restricted to a tiny number of processes run in one current block reporting their results to processes being run in subsequent blocks. It would suffice to find the sweet spots in the parameter space of a Markov-Chain Monte Carlo algorithm, but I haven't thought of a way to generalize it beyond that. [4] http://tinyurl.com/7dtt536 (archive.org copy of CPUShare blog post.)