White King, Red Queen: the matrix. Part VI.

They are coming.

In 2009 Hiarcs 13 on a smartphone dominated a world-class chess tournament with humans. At latest in the moment, it should have became clear that the world of computer chess should get separated by the world of human chess.

Indeed they were: as an example, in 2006 a group of compute chess enthusiasts founded the CCRL rating list, dedicated to testing of chess engines in controlled hardware conditions. They would donate their computers to let engines playing against each other just for the pleasure of compiling a list of the strongest chess computers. This list, still maintained after 12 years, contains 353 engines as of July 22th, 2018, many of them tested in multiple versions.

The arena

We have seen it often: if an appropriate, fair arena is set up, with a meaningful reward, people will gravitate towards it and, given time, the arms race will begin. In the 2000ies, many of such arenas were being made available, in the form of rating lists, computer chess tournaments and the like. The arena was set. As for the reward, chess engines were now very hot even for professional players and you could still sell them as programs for up to 100 $. If you think that you can squeeze a chess engine under 10k-20k lines of code, and you only need to comply with a specified protocol, it sounds like the perfect challenge for algorithms enthusiasts. The realm of nerds, so to speak. And the nerds came. In few years, we had many engines fighting for the title of strongest chess program: Rybka, Houdini, Komodo. And then came Stockfish.

The crowd

Until 2013, Stockfish was a good, but not top open-source chess engine. Until Gary Linscott, one of the contributors, constructed a distributed test framework for the engine. In this framework, anybody would be able to write modifications of the code and test this against the currently best version. How would this be tested? People from all over the world would donate their machine and a program would take care of let those computers playing games of the proposed modification (called patch) against the current best version (called master). After some dozen of thousands of games, it would be decided statistically whether the modification could be applied. A round of human based checks for code quality and you could get your patch included into master in less than a week! In the first 5 years of its life, Stockfish had 16 people contributing code. In the second 5 years, 94. Furthermore, since the patches would be tested with some very high predefined criteria, the risk of introducing errors just dropped to almost 0. Result: in less than 1 year Stockfish was sent to the top of all rating lists.

The arms race

One further piece to the puzzle: Stockfish is an open source engine. This means whoever can go and read the source. Furthermore, all discussion is done in public in a google forum, so everybody can know the background of decision and design. On one hand, we have the sheer amount of ideas and computing power available on the its framework. On the other side, these ideas are public, so all proprietary engines can go a dig for ideas which will work also in their context. In the subsequent years a true arms race between three competitors, Stockfish, Komodo, Houdini, followed. They would continuously exchange the crown of the strongest engine in the world, informally assigned as the winner of the last TCEC tournament. This continues until today.

Clouds on the horizon

Some of you will have known that chess engines were against on the main page some weeks ago. A team from Deep Mind, the AI subsidiary of Google, trained a neural network able to compete with Stockfish on custom hardware. The history repeats: first a year-long quest to have a alpha-beta engine beating the human chess world champion and its biological neural network. Now a year-long quest (to come) for an artificial neural network to regain the crown. We'll see how it ends! In the meanwhile, an open source team is trying to replicate the results of the Google team in an effort call Leela Zero. Again, one of founder of this project is Gary Linscott, the former Stockfish maintainer which created its testing framework.

Why the matrix?

Interestingly, now the challenge is not anymore between persons writing programs. But between engines and their development paradigm. In a way, the engines are using us, the humans, to improve themselves such that they still will be operated in the chess engines arena. As I said: the matrix.

Is free will a thing, actually?

If he doesn't have free will, who does?

I always read backreaction with pleasure and one of the latest articles attracted my attention.

It was about free will. Free will is cool, really. This is a slightly, but only slightly, religious concept, so people from all faiths will discuss about it without much arguing. But it also philosophical, so hard core atheists will join the discussion. It has something to physics, since somebody in the room always bring up determinism. Everybody has it (or nobody, depending on who you ask) so everybody is going to discuss it. In the end, it does not matter that much, so discussions do not become too heated. And finally, it is the main topic of The Devil's Advocate, which is a great movie. I love it. Both free will and Al Pacino, of course.

Free-will has nothing to with determinism

That said, I would like to offer my humble contributions to the topic. The first objection to free will which always arise is determinism. What is determinism? The belief that if I would know position, velocity and everything else of all particles in the universe, then you would be able to predict everything which is going to happen in the future. It looks like this strong form of determinism would destroy free will. The first clear formulation in western philosophy of this is due to Laplace. Luckily enough, we don't have to discuss this because it is probably non-true. First, it is non practical due to the amount of chaos available in the world: small perturbations lead to large changes.

It is plainly wrong in the standard interpretation of quantum mechanics, since wave functions collapse contains a randomness. So, no physical determinism. 

Also, if determinism has anything to do with inference, it has been shown that you cannot do complete inference in systems where you can perform standard logics. Interestingly, this mathematical proof requires Cantor's diagonal argument. Which one of the single most important pieces of maths, so make sure to follow the Wikipedia link, please. I will come back to this in the end.

Given that determinism is wrong, we can set it aside in discussing free will. Yes, in a deterministic world, there is no space for free will. In a non deterministic world, we do not know. Maybe we are free, maybe we are random. Maybe we are merely non-computable.

Free-will has nothing to do with predictability of human choices

In the later years, there has been a copious amount of papers about predicting human choices before the reach consciousness, due to continuous improvement in brain imaging methods. See here, for example. This has caused a lot of stir among people. I do not know exactly what about: the will can well be free even if it is not conscious. You can make a free, conscious decision with your own will, but your will could also make a free decision without notifying your consciousness organ in advance. Free will is not the same as rational choice, in the end.

What exactly is free in the free will?

This is the very tough question: I think one important point is the capacity of influence its own choices and the ability of (consciously or not) question one's first (or second or third) guess. In general, when we talk of something which is free, we mean that said something is able to initiate actions out of its own initiative and it is not completely determined by the external forces. We talk about freedom even in physics: how many degrees of freedom has a body? In how many way it can move freely without being constrained? I would claim that free in "free will" denotes the mind's ability of making conscious or non-conscious choices not completely determined by factors external to the mind itself. Similar to a body with a positive number of degrees of freedom can actually move, although it is influenced by external forces and the own inertia, a free-will can pick choices, although these choices are influenced, to large degree, by external factors.

Gödel, our old friend

We already have seen that Mr Wolpert used Cantor's diagonal argument to show that there is no mathematical free will. You cannot say Cantor without's saying Gödel. Gödel's first incompleteness theorem states that any axiomatic system which is has a model of arithmetics will contain a mathematical sentence which is neither true nor false. If you additionally impose that the arithmetics should be standard (in some very specific sense), this sentence will true but non provable. In other words, free will seems to be related to Gödel and to the very nature of natural numbers which is itself quite dubious. By the way, Gödel's theorem proof heavily uses Cantor's diagonal argument, so we are back to where we started.

Summing up

Summing up, we maybe have free will, but then again no. We'll see. In any case, if natural numbers are actually true, we will have to pick infinite number of axioms to avoid problems. And this means a lot of choices, and all of them are free, unless you use standard arithmetics. But you don't need to. Good evening to you.

In praise of laziness - in math and in life

 Since around 2000 I have pursued a career in math-related fields. I studied maths, I did my PhD in functional analysis and PDEs, then I moved to stochastic processes. After leaving academia and joining industry, I happened to be involved in a project about non-linear structural mechanics.

All in all, at my venerable age of 37, I am satisfied to have a google scholar profile and I am very grateful to have enjoyed collaborating with many people much more gifted than me.

As years pass, I become more and more convinced that the only way I could achieve this was not mathematical talent but sheer laziness.

Some background on why I was good in math, but not really

I had almost no math education until the age of 18. I went to a humanistic school, learned Latin, Greek, Philosophy and the like. In my last year, I heard rumors about limits and exponential functions from my math teacher. I thought them esoteric mysteries reserved to University students. I decided, this was not interesting enough for me.

However, I was somewhat talented for numbers and I still can calculate very quickly possible figures, on the other side I have never had any intuitive grasp of geometry. I literally hate triangles and congruence criteria, not to mention 3D geometry, which should be prohibited by UNICEF as a perverse mean for teens torture.

So I went to University, and started studying mechanical engineering, just because of good job perspectives, if you want to know it. However, after hearing the introduction of natural numbers according to Peano, I decided to switch in the end. I just enjoyed that so much!

How to survive pages of estimates through laziness

Somehow I got a master degree in math. I was 25 as I finished my studies, I decided that I still could some fun time and did my PhD. Laziness is no good starting point to complete a PhD in maths, though. But it has some advantages. Whilst navigating the treacherous waters of choosing a cool topic for my thesis, I steered clear from everything what promised years of suffering because of excessive calculations. In contrast, after some attempts I settled to a reasonable subject where more or less nothing had be written, so that I could graze on green pastures and produce interesting theorems with limited effort. It was not a conscious effort in that time, but now, years later, I recognize the pattern very clearly.

I had a very good supervisor. He was back then PostDoc in my institute, and he had quite some talent for doing long calculations without errors. He tried to motivate me to learn this skill. Again, I was very bad at this, but had some insights for finding shortcuts and proving theorems with as little effort as possible. Laziness, again. This saved my academic life in similar situations over and over again. Another example: I was once included as an author in my most cited paper for the only reason that I revealed to a very good PhD student the secret of shortening calculations with the judicious use of Fourier transforms and matrix inversion. 

Laziness in life

In the end, I think that laziness helped me in life too. We built a house with my wife some years ago. This is quite a complex endeavor, I can say you. It needs hundreds of decisions, and can become seriously exhausting. Somehow, we decided to settle for a "good enough" strategy. When faced with any decision or option we would always take the first one which was good enough. We talked to friends in the same situation. They were horrified that we planned, chosen and bought our new kitchen in well below 4 hours. When I was a PostDoc, I was struggling to start an academic career. I changed field after the PhD, so I probably would have needed some more stations before becoming settled. Judiciously, I just decided that I would give industry a chance, and I have never regretted this. Academia was just not my path. By the way, the PostDoc which supervised me during my PhD thesis is now a respected Professor. It was his path.

I admit, there are situations in which you have to push and go the extra mile. Can you ever be sure that this is the case now? Burnt energies do not come back for free and if you rush too soon you will not do the run.

And you? Have you already found yourself in situation when healthy laziness helped you succeed?

Everything is a team sport

 I have been reading this article on the Rebels At Work tribe site and I would like to share my thoughts on diversity, but taking a different vantage point for looking at this. Let me start with two insights:
1) whatever you do is a team sport and
2) you have to understand the game you're playing

Let me give some example before I dig deeper in this concept.

Example 1: Complex Tooling

Imaging you want to build the tool to design your next cool device. It will need some physics, stats, CAD, circuit design. You're trying to write something like a scientific library. This is a team sport: specifically, it feels like basketball. Since very in-depth discussion are the bread-and-butter here, you need small teams in order not to get lost in group dynamics; also, since in-depth knowledge is very rare, it almost never pays off to have duplicates. If you are trying very hard to get this library for doing multiphysics modeling of microdevices, you dont want to have two fluid dynamics experts and no software engineer.  You would miss one discipline badly. For every role on your team you need exactly one person: one point and one shooting guard. One small and one power forward. One center. You need them all.

Example 2: Hardware Development

You now have your tool to design your devices, with very cool physics implemented, you can do CAD and whatever else. You want to bring now to production. Your team will grow. Deadlines will now feel more important than technical depth. You need redundances; on the other hand, do you really need to have production engineers and software testers in the same meeting? Probably not, so you can enlarge your team, and you speficically need coordination. It feels now more like a football team. You have a goalkeeper, 3-5 defenders, 3-5 midfielders, 2-3 strikers. You typically need a deep-lying midfielder, the project manager, purely in charge of keep the ball moving. Also interestingly, you have reduced interaction between separated roles: although the occasional pass from defense will serve a good scoring chance for a talented striker, this is an exception.

How do I recognize which game I am playing

As usual, there is no easy recipe. I find mapping your environment can help you. If you are doing research on something very innovative and technically challenging, you probably will need small teams to encourage in-depth discussions: we are talking about basketblall. If you are in product development, and you need to meet deadlines, but the technical challenges are solved, maybe you need a football team with redundances.

I think using this model can help you investigate two important aspects: the amount of necessary internal communication and the right level of diversity.

More or less communication?

Imagine a football team in which the goalkeeper always tries to pass to the forwards; passes will be not precise, the goalkeeper will not even see exactly what is going on on the other side of the field. If passes are the communication in ball games, and we use this model in our product development team, it seems that too much of communication could even be detrimental in a product development team. You need the midfielder who let the ball move from defense to offense, so to speak. Ever asked yourself if you really need a weeky meeting with all 50 members of the team?

On the opposite, imagine a basketpall team in which the shooting guard plainly refuses to pass to the center: it sounds like the team is doomed to fail. In small teams with tough challenges is the same. You sometimes really need the combined skills of both the avionics expert and of the software tester to get the reliable library you really need to fine tune your airplane wing!

Right level of diversity

Accordingly, you can use this model to understand on which level to address skill diversity in your team; it does not make sense to insist in having two central defenders with very different passing skills. It does not hurt and it does not help, so you're free to choose using other criteria. On the other hand, make sure to have some defenders and some midfielders and some strikers. Yes, and more than one player which can kick penalties will not harm.  The same at work: in a product dev team, make sure to have some redundancy in every field, but don't get to stressed about lacking skill diversity inside of a position. Lack of diversit will kill you in the mid-term, though, in the research team, as soon as you encounter a problem outside the range of the people there. Ah yes, and they will not even notice that they have a problem, probably.

Hope this helps you, feel free to comment and share you're experience!