Read the article. Python doesn't scale well under load but go does. Rather than go with an alternate python runtime, or rewrite everything in something else, they wrote a python runtime for go.
I don't quite believe that number. It would imply that you would be capable of serving google's ad load in ~2012 from 100 or so machines, which, to be honest, I don't believe for a second.
Serving from 100 machines is much harder than serving from one maching. First, it implies that working set is much larger (certainly, Google has 1000 times more unique ads).
"Single" ad could be served at 1000000 rps with handcrafted C from single machine (look at techempower benchmarks).
10000 unique ads could be easily served at 10000 rps from single machine.
But to serve 10000000 unique ads one should implement really complex optimizations, and throw a lot of computer power into.
"Single" ad could be served at 1000000 rps with handcrafted C from single machine (look at techempower benchmarks).
This of course depends on the type of add. That gives you what, 2000 cycles per ad, which if the ad is just a few characters is doable, but if it, for example, contains a 64*64 image, you've used up that many cycles moving the image.
It doesn't change my point: "5000 ads per second with dual-core server" could be either "fail" or "win", depending on total numbers of unique ads and complexity of ads selection logic.
As someone who doesn't use Go or Python: what is the use for this?
rjcarr 6 points · Jan 2017
Read the article. Python doesn't scale well under load but go does. Rather than go with an alternate python runtime, or rewrite everything in something else, they wrote a python runtime for go.
ahmadalhour -19 points · Jan 2017
I work for a company where everything is developed with Python:
crawling pipeline
data engineering pipelines
Public APIs
And our public API endpoints get hit in the 200-300 millions hits per month.
So you are extremely mistaken about Python not scaling well under load. You just don't know how to scale it.
Full disclaimer: I work with TrustYou.
[deleted] 43 points · Jan 2017
300 million hits per month is about 114 hits/second.
I think you and Youtube have different ideas of what "under load" means.
[deleted] 22 points · Jan 2017
[removed]
zardeh 5 points · Jan 2017
I don't quite believe that number. It would imply that you would be capable of serving google's ad load in ~2012 from 100 or so machines, which, to be honest, I don't believe for a second.
funny_falcon 5 points · Jan 2017
Serving from 100 machines is much harder than serving from one maching. First, it implies that working set is much larger (certainly, Google has 1000 times more unique ads).
"Single" ad could be served at 1000000 rps with handcrafted C from single machine (look at techempower benchmarks).
10000 unique ads could be easily served at 10000 rps from single machine.
But to serve 10000000 unique ads one should implement really complex optimizations, and throw a lot of computer power into.
zardeh 1 points · Jan 2017
This of course depends on the type of add. That gives you what, 2000 cycles per ad, which if the ad is just a few characters is doable, but if it, for example, contains a 64*64 image, you've used up that many cycles moving the image.
funny_falcon 1 points · Jan 2017
Single machine is not single cpu-core. Even Intel-cpu based server may have 24*8=192 cores (384 with hyper-threading) http://ark.intel.com/ru/products/93790/Intel-Xeon-Processor-E7-8890-v4-60M-Cache-2_20-GHz?_ga=1.162292223.210121279.1483966676
zardeh 1 points · Jan 2017
funny_falcon 1 points · Jan 2017
It doesn't change my point: "5000 ads per second with dual-core server" could be either "fail" or "win", depending on total numbers of unique ads and complexity of ads selection logic.