O_CREAT

Stay Away From Bitcoin Arbitrage If You’ve Got A Heart Condition

This must be true for most financial activities. But since I’m receiving something like 5 emails per day from random people [1] who wants to run my bitcoin-arbitrage tool, most of the time I’m replying with some warnings:

• Can you trust the exchanges ? The answer is NO, you can’t:

  • Trade engine can be laggy (up to 1 hour lag).

  • APIs are buggy. get-order-book replying with 2 hours delay is unacceptable. some API function doesn’t work time to time

  • Bad “cloudflare” config sometimes blocks normal API calls (Yes, I’m looking at you bitstamp)

  • The site closes but a part of API is still working (Yes, i’m looking at you bitcoin24)

  • And the worst thing: they can close at every moment and block your money (1 hour to infinity). History prove this one, see what happened to Bitcoinica, Bitfloor, Bitcoin24, Bitcoin-Central. Who’s next ?

• Can you trust tools ? Of course NO. If you can’t read code (blind, not a developer or closed source), don’t use the tool. A malicious dev can steal your money so easily.

• To trade you need money, fiat AND bitcoin. If you don’t have bitcoins, are you ready to buy some ? Yes, risks everywhere !

• Moving fiat is so long… SEPA transfers ? sure let me wait a week (I’m wondering why it’s so slow, do banks have computers ?). That means you’ll need A BUNCH of money (fiat AND bitcoin) in every exchanges you’re trading on. Then balance them time to time when they drift. And that opens a big risk if one exchange closes and blocks your money.

So yes, Bitcoin arbitrage seems easy and very lucrative, but it’s very risky. Not because of arbitrage principle itself, but mainly because you can’t trust exchanges.

[1] random people includes: developers (who wants to rewrite the tool in Ruby, Java, C, Javascript, Python 2.x), students (finance, math), home trader (that one is scary), unknown (“hey, i stumbled upon your program, I’ll give you $500 to make the setup on my computer”).

Redis ZSET Underlying Datastructure

Sorted Sets: ZSET

Redis ZSETs are a very good to store ordered data like leaderboards:

• You need to get the current score for an userid (every SQL or NOSQL can do that efficiently)
• You need to get the current rank for an userid
• You need to get a list of pairs userid/score sorted by score

Datastructure

In the Redis documentation, we can read complexity is O(lg(n)) to ZADD / ZRANK / ZREM and O(1) for ZSCORE. How are they implemented ? O(lg(n)) you said… hmmm smells like a tree or binary search. ZSCORE (a kind of search) in constant time… hmmm smells like a hashtable. So how is it implemented ? a mix of hashtable and tree ? No, but that’s close, it’s a mix of hashtable and Skip List. Skip list are an alternative to balanced binary tree.

You should read the original Skip List paper.

From the redis documentation:

ZSETs are ordered sets using two data structures to hold the same elements in order to get O(log(N)) INSERT and REMOVE operations into a sorted data structure.

The elements are added to an hash table mapping Redis objects to scores. At the same time the elements are added to a skip list mapping scores to Redis objects (so objects are sorted by scores in this “view”).

So, how to perform a ZRANGEBYSCORE in linear time ?

1. search the min element. O(lg(n)) complexity (with n=|set|)
2. start from min and follow links to the next element until you reach max. O(m) complexity (with m elements returned)

Also, there is a modification from the original skip list implementation, level 1 list is doubly linked, so it gives you: ZREVRANGE, yeah !

Algorithm Complexity

I started to code some random problems from Project Euler. To have an extra hint of which kind of algorithm I’m looking for, I made the following table.

Read it like: “Assuming a computer can do 10 000 000 operations/sec, with a O(n^2) algorithm, you can solve a n ~ 3000 under a second”

Complexity      |   n for 1 second  |   n for 1 minute
------------------------------------------------------
O(1)            |   inf             |   inf
O(lg(n))        |   huge            |   2^(10^7)
O(sqrt(n))      |   10^14           |   36*10^16
O(n)            |   10 000 000      |   600 000 000
O(n*lg(n))      |   526 172         |   24 446 750
O(n*sqrt(n))    |   46 415          |   711 378
O(n^2)          |   3162            |   24 494
O(2^n)          |   23              |   29
O(n!)           |   10              |   12

Project Euler problems should be solved under 1 minute (even in Python, most problems I solved has been < 2 secs), so if the input N is ~10 000 000, you should look for a O(n*lg(n)) solution or better.

Also if you’re developping Python code, you should have a look at these Python Time Complexity Tables. I think other language libraries use the same underlying datastructures and algorithms.

List redis keys, filter some of them, then hget on this filtered keys

mashimeb:

$ echo "keys *"|redis-cli|grep user|xargs -I % sh -c "echo hget % name |redis-cli"

Bitcoin Arbitrage

I update my bitcoin arbitrage project: https://github.com/maxme/bitcoin-arbitrage

More modular and now include trading bots if you want to automate buy and sell orders on some exchanges.

Bitcoin Arbitrage Opportunity Watcher

I released a quick hack to watch for bitcoin arbitrage opportunities. Code and documentation are on github: https://github.com/maxme/bitcoin-arbitrage

I made some bitcoins with it, pushing coins from mtgox to bitcoin central and back. If you are crazy enough to plug a API-trading tool, you should be one of the few bitcoin automated arbitrage trader.

Start A Development WebServer - One Liner

Run a webserver at http://localhost:8000 serving current directory (and subdirectories)

$ python -m SimpleHTTPServer

(Source: mashimeb)

Emulate A Mobile Device In A Desktop Chrome

Another tip to debug html5 apps for mobile on your desktop: install the Ripple Chrome Plugin

Allow Chrome to do Cross Domain HttpRequest

Very useful to debug webapps:

$ /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --disable-web-security

Game Design Lesson - Super Mario Bros (NES)

I read this very interesting snippet in the Wikia Super Mario Bros’ page

“Interestingly, in the very first portion of World 1-1, the developers designed it so that the a newcomer almost always gets a Mushroom. In the first level, there are blocks that the player goes under. A menacing Goomba approaches the player, and instinctively the player jumps over it. By the time the player reaches the Goomba and jumps, they will hit a ? block above that would reveal a mushroom. The mushroom goes to the right, hits a pipe and comes towards the player. Since the mushroom resembles the Goomba, the player thinks to jump over it again. Doing this, however, will almost always lead the player to jump right into the Mushroom since after they jump they hit another block from above which causes them to come back to the ground and hit the mushroom. This was to teach players that Mushrooms were a positive thing in the game.”

(Source: reddit.com)

“Curiosity What’s Inside The Cube” - Game Maths

Every game developer heard about the last Peter Molyneux game “Curiosity”. Sorry, it’s not a game, but a game experiment. Bad graphics, bad UI, bad music, but a web-2.0-like curious community. Yep, it’s like EVE Online but not like World Of Warcraft. It’s a “one world instance”-game. All players share the same world.

Another community, Wikipedia authors, worked on game maths behind the game and are now able to estimate time to completion:

Curiosity Wikipedia Article

I predict one thing: a big fail when the cube will be nearly finish. The game already suffers connexion issues at launch.

POST 10 http requests to “http://te.st” (fake) server, maximum 5 in parallel, data POSTed is id=1, 2, …

mashimeb:

$ seq 1 10 | xargs -t -P 5 -I % curl -s --data id=% http://te.st/ > /dev/null
curl -s --data id=1 http://te.st/
curl -s --data id=2 http://te.st/
curl -s --data id=3 http://te.st/
curl -s --data id=4 http://te.st/
curl -s --data id=5 http://te.st/
curl -s --data id=6 http://te.st/
curl -s --data id=7 http://te.st/
curl -s --data id=8 http://te.st/
curl -s --data id=9 http://te.st/
curl -s --data id=10 http://te.st/

Shell script one liners

Some time ago, I wanted to share my shell mini-scripts and tips, so I started this: http://mashimeb.tumblr.com