Ytreza wrote: ↑Sun Sep 22, 2019 1:34 am
Yes after playing a bit with the code I realized by myself that this method couldn't be blindly applied to hnefatafl. Actually it already takes a very long time to get an acceptable AI playing connect4.
I think it might. At least if you have a few millions to spend on hardware and power.
Some people like to just throw more computing power at a problem, but I'm none of them.
Ytreza wrote: ↑Sun Sep 22, 2019 1:34 am
As I wrote in my previous post, I have no experience at all in AI programming, neither from the point of view of computer science nor in maths.
I can help with that. From a mathematical standpoint it's just awful anyways. I mean seriously.
We have no idea whether something similar happen if try two times. The practical results just give a notion of "usually".
But if you have questions, I'd be glad to help.
Ytreza wrote: ↑Sun Sep 22, 2019 1:34 am
(I know C,C++ and python), but there is nothing as crazy as DNN.
That's more than enough for this project. All the heavy lifting stuff should be written in C or C++. I have build NN in low level languages by hand for years, but with tensorflow (tf) there is no really reason to do that anymore. It's better than everything I've ever build and in my eyes the real reason to use python. It's at least the real reason I'm dealing at all with this ... language.
That being said, we should be able to get nearly all of the stuff we need from higher level framework keras (which is just tf with a small wrapper).
It still helps to understand what you are doing, but it essentially safes us the hassle to implent own DNN stuff. It's not the hardest thing I've done in my life, but it's very error prone.
Ytreza wrote: ↑Sun Sep 22, 2019 1:34 am
I'm afraid I would not have time to dedicate myself to another big project as you seem to suggest it is. But maybe working together we could get somewhere?
It's similar on my end. I can't make this my number one project right now, but that doesn't mean I can't invest some time.
Ytreza wrote: ↑Sun Sep 22, 2019 1:34 am
It is surely tempting. Also, if we obtain a fairly good AI we could think about writing a scientific paper.
Let's start by building one. The question is whether our approach make it stronger, not whether it gets decent strength. But either way we have to build it first.
Ytreza wrote: ↑Sun Sep 22, 2019 1:34 am
1. I didn't understand what you mean by "you should three three input vector for all fields". By "fields", do you mean "squares"?
Yes, sorry. I should have stuck to one notation.
Ytreza wrote: ↑Sun Sep 22, 2019 1:34 am
2. [...]
Therefore I think that one could set a threshold of 100 moves above which black wins (100 moves total, with or without captures).
I think that won't be the most relevant question. At the same time I've seen that games can stay interesting for quite a long time, if both sides do top notch moves. Usually humans make mistakes, because humans suck...
But we can start with 100 moves in total, sure.
Ytreza wrote: ↑Sun Sep 22, 2019 1:34 am
I agree that edge forts don't seem to be so important. However their importance could be revealed by the AI (see below).
There is a difference between the scenarios.
We will loose a ton of computing power before the ai is able to play strong enough to threaten a total beginner. It's important to make the stuff as simple as possible when we start out. I'd like to instroduze new concepts to this ai as we go. It's much faster to take an ai with basic understanding of the game and teach edge forts, but to learn a new ai. That being said it's hard for the ai to notice edge forts at all. It's not mainly about the 100 moves. It's more about the fact that it takes a long time for the ai to figure that out, because you need A LOT of luck to build an edge fort by accident. And you need to do that many times for the ai to figure it out.
Ytreza wrote: ↑Sun Sep 22, 2019 1:34 am
3. I don't get the point (I don't understand " It's helpful for the network not have to learn the difference between white and black moves, so it makes sense to include both in the output layer.")
Hnefatafl is so interesting, because it's a very asymmetric game. For that every reason we should differentiate the output nodes and designate individual ones to every kind of piece we have in the game. We have just three types, but it makes it simpler, because it gives a sort of meaning the the individual move.
Ytreza wrote: ↑Sun Sep 22, 2019 1:34 am
5. Yes, python is slow. I'm not used to ffi but I don't think it'd take much time to learn, and it would be useful in any case for other projects.
If you want to continue to use python, it's helpful. If you want to do reinforcement learning it's great. I find it a chore (like almost everything in python).
Ytreza wrote: ↑Sun Sep 22, 2019 1:34 am
A question I keep asking myself about hnefatafl strategy is should white really try to escape, or rather capture as much black pawns as possible to get space to build an edge fort?
That question isn't reasonable inherently. Because you can do neither if black doesn't want to allow it. Black allows you to attack a certain corner or to take a pawn. The point is that while it's very easy to defend a single corner or keep all pieces, doing all at once is hard.
In my personal experience having a pawn more or less is not the important part. The total amount of pawn is usually not the issue regarding edge forts. It's much more the systematic weakness of two adjacent corners that bind all the available pawns that are now missing to provide even basic protection the the edge.
As much as I'd like this to be answered by an ai, that is nothing an ai will tell us 2019 or 2020. There are a lot of simpler questions that need to be answered first.
Ytreza wrote: ↑Sun Sep 22, 2019 1:34 am
Most games I see (my games but more importantly games between high rank players in the archive) end because white or black chose a wrong variation at some point. The questions I expect an AI to answer are:
i/ Is there a winning corner attack for white? (100% chance of winning). This would be very disappointing of course (the games would end in the opening).
I doubt there is a simple solution. But that is indeed something I'd like an ai to have a say on. At the same time it doesn't prove anything, it just suggests...
I think the other questions are interesting, but at least for now totally out of scope.
Ytreza wrote: ↑Sun Sep 22, 2019 1:34 am
I think (tell me if you don't agree) that when white manages to capture 6 attackers, he has good chances in endgame even if the corners are closed. The reasonning behind this number is as follows: black needs 12 pawns to close the corners, so if white captures 6, black is left with only 6 "movable" pawns. If white made 6 exchanges in middle game, there are 6 white + powerful king VS 6 black in endgame fighting to build/prevent an edge fort.
In this case, the games would be very interesting with a lot of long-term strategy involved (not only computation of tactical variations for corner escape).
That's why I said above that edge forts could be very important at some level of play.
I don't think that's such a mayor problem as you make it out to be. Think about 16 black pawns bound in the corners. That means that white can choose any corner and position 5 pawns at VERY specific squares and the sixth pawn further defending the exposed pawn and the king inside the castle. That means that upon the symmetry of the board there are just two possible positions for the white pieces (king at the corner and king one square away). Even though black has just two pawns to defend that's not really that scary, even if you have just two free pawns to defend. The point is that the white edge forts get much scarier if white manages to get 4 connected squares at a single edge. But if black can prevent that...
Ytreza wrote: ↑Sun Sep 22, 2019 1:34 am
Anyway, I'm afraid I don't have the capabilities and time required to program this nice stuff for now, but I really hope someone will do it someday! From a newbie point of view, it doesn't seem so hard for an expert (ok, harder than just feeding alphazero with the rules, but still...)
A decent movegen on it's own, isn't easy already. Tf isn't that hard, but setting up the flow graphs is still a chore. Yes, I do notice I use that word a lot when it comes to python. We haven't talked about the graph. After all the estimation of depth one will never be *that* great. That's true for Hnefatafl much more than it's for other games like chess. Once the two components of training and producing training data are well encapsulated components, that can be executed asymmetrically, the network stuff should be fine. After all we don't even need to compress the stuff very well. Even three bytes per move would be fine (even 10 bytes shouldn't be a problem).
Let's formulate the plan on a very high level:
1. build movegen (I sort of have one)
2. use the movegen to feed some tf graph
3. feed that into some classical data structure (so the engine is able to "inspect" positions)
4. use the classical recursive reinforcement learning to update the DNN
(5. do that as a distributed system)
Regards
nath