So far the minimal GTP command set includes:
Boardsize Komi Fixed_Handicap White Black Genmove_Black Genmove_White Undo (* provisional for now) Quit
What else?
Would it be useful to know whether the engine thinks the game is over and perhaps thinks itself the winner? IE handing the scoring phase, perhaps including declarations of the life/death status of groups? The commands you've listed allow a game to be played through a smart referee, presumably a human, who could handle cases where one engine thinks it has killed a group but another has not (ie an engine bug) and who could really determine such things as the game has endend. Without these smarts, full automation of a set of games (for learning or competition) may not be possible. For instance I've seen instances where an engine is trying to make a move the other thinks isn't possible. This happens in programs using machine learning techniques, depending on how much expertise is coded into the engine. Handling automated game playing required us in OpenGo to put intelligence into the 'referee' to cope with this -- thus the protocol required some way to tell an engine that it's move is rejected and the game is over (it can cleanup or do what it has to do then)...
Taking a step away from gnugo itself, are these issues you want to concern yourself with?
Also a question: does the protocol include the negotiation of these terms with the engine (ie a request for it's capabilities in the first place, and an ack or something implying it can play to the terms requested) (sorry if this should be obvious in the documentation, but i've not really gone through it...) jeff
Phil wrote:
So far the minimal GTP command set includes: Boardsize Komi Fixed_Handicap White Black Genmove_Black Genmove_White Undo (* provisional for now) Quit What else?
Jeff wrote:
Would it be useful to know whether the engine thinks the game is over
Then it should answer a genmove_* command with PASS.
and perhaps thinks itself the winner? IE handing the scoring phase, perhaps including declarations of the life/death status of groups?
Yes, I think we should have optional commands to ask the engine about its opinion on score and group status at the end of the game. Such commands are implemented in GNU Go as final_score, final_status, and final_status_list.
The commands you've listed allow a game to be played through a smart referee, presumably a human, who could handle cases where one engine thinks it has killed a group but another has not (ie an engine bug) and who could really determine such things as the game has endend. Without these smarts, full automation of a set of games (for learning or competition) may not be possible.
The referee could also use a program, e.g. current versions of GNU Go would do a decent job at automatic scoring. To make it more clear what I mean I have the following setup in mind:
A <-- Arbiter --> B | v scorer
The arbiter communicates with engines A and B through GTP channels and relays moves until both program passes. Then it opens a third GTP channel to another engine to which it passes the position and asks it to score.
Furthermore it's of course always possible to use strict Tromp-Taylor rules and let the engines suffer their mistakes if they fail to remove dead stones. In that case the counting could easily be coded in the arbiter.
Anyway it's true that the protocol doesn't try to implement any agreement phase. If someone can come up with an elegant design of this it may be included in some later revision of the protocol, but hardly in the first one.
For instance I've seen instances where an engine is trying to make a move the other thinks isn't possible. This happens in programs using machine learning techniques, depending on how much expertise is coded into the engine. Handling automated game playing required us in OpenGo to put intelligence into the 'referee' to cope with this -- thus the protocol required some way to tell an engine that it's move is rejected and the game is over (it can cleanup or do what it has to do then)...
Taking a step away from gnugo itself, are these issues you want to concern yourself with?
Maybe, depending on whether I understand you correctly. In my opinion the arbiter has two reasonable responses for handling differing opinions on legal moves:
1. Stop the game and declare a loss for the program breaking the rules. (Either by determining this itself or asking a human to investigate.) 2. Stop the game, let a human operator fix the bug causing an illegal move to be made, and restart the game.
If an engine does need help identifying legal moves, this can be done by letting it open a GTP connection to a program with this capability, which it can ask about move legality. This would require some command like is_legal or all_legal, which are available in GNU Go. Probably these commands should be standardized but not required from every engine.
Also a question: does the protocol include the negotiation of these terms with the engine (ie a request for it's capabilities in the first place, and an ack or something implying it can play to the terms requested) (sorry if this should be obvious in the documentation, but i've not really gone through it...)
Again I'm not fully certain I understand the question, but one of the basics of the protocol is that a command from the arbiter is answered either on the form "=[id] result" or "?[id] error_message", where the former case indicates success and the latter case failure.
The only exact specification of results and error messages is currently the GNU Go source code. This needs of course be addressed by the standard. I'll try to specify the syntax and semantics of all the basic commands as they are understood by GNU Go in a later message.
issues like cr & crlf can be finessed by requiring an end statement, such as semi-colon, so that cr & crlf can be ignored. but then it isn't quite so elegant. but then what is the goal of the text: ie it seems that you want some sort of readability in the protocol, but why?
Primarily for two reasons: 1. Make it easy to debug the protocol, e.g. by inspecting a log to see what part failed to follow the protocol if the communication breaks down. 2. GTP commands can be written as scripts for regression testing, which was the first application of GTP in GNU Go. These test suites are more easily maintained if the protocol is readable.
/Gunnar
Anyway it's true that the protocol doesn't try to implement any agreement phase. If someone can come up with an elegant design of this it may be included in some later revision of the protocol, but hardly in the first one.
I believe it's possible to hide all of this complexity within the interface software (the arbiter.) There can be negotiation without the engines needing to understand complex protocol.
I think our goal is to provide just enough funtionality in the protocol itself that the interface software can be arbitrarily complex without making it difficult for the engine programmers to comply.
I don't think we need a 3rd engine to do scoring but the point is that you CAN do this if you want to. It's really up to whoever implements the arbitration software to decide, and I would just consider the 3rd program part of the whole arbiter software package (indeed, why not just link gnugo into the arbiter executable?)
So what I'm suggesting is that we have a rich enough protocol that we can support anything we want LATER, and by design. The protocol is a low level thing, give it enough flexibility to get anything it needs from the engine and we can build whatever we want to later.
I don't think this conflicts with the goal of simplicity, because I think there are very few core things needed. Technically, we could probably get by with 5 commands, but we might be far happier to include another 6 or 7 that will be incredibly useful later.
What do you think?
Don
Jeff wrote:
Would it be useful to know whether the engine thinks the game is over
Then it should answer a genmove_* command with PASS.
and perhaps thinks itself the winner? IE handing the scoring phase, perhaps including declarations of the life/death status of groups?
Yes, I think we should have optional commands to ask the engine about its opinion on score and group status at the end of the game. Such commands are implemented in GNU Go as final_score, final_status, and final_status_list.
The commands you've listed allow a game to be played through a smart referee, presumably a human, who could handle cases where one engine thinks it has killed a group but another has not (ie an engine bug) and who could really determine such things as the game has endend. Without these smarts, full automation of a set of games (for learning or competition) may not be possible.
The referee could also use a program, e.g. current versions of GNU Go would do a decent job at automatic scoring. To make it more clear what I mean I have the following setup in mind:
A <-- Arbiter --> B | v scorer
The arbiter communicates with engines A and B through GTP channels and relays moves until both program passes. Then it opens a third GTP channel to another engine to which it passes the position and asks it to score.
Furthermore it's of course always possible to use strict Tromp-Taylor rules and let the engines suffer their mistakes if they fail to remove dead stones. In that case the counting could easily be coded in the arbiter.
Anyway it's true that the protocol doesn't try to implement any agreement phase. If someone can come up with an elegant design of this it may be included in some later revision of the protocol, but hardly in the first one.
For instance I've seen instances where an engine is trying to make a move the other thinks isn't possible. This happens in programs using machine learning techniques, depending on how much expertise is coded into the engine. Handling automated game playing required us in OpenGo to put intelligence into the 'referee' to cope with this -- thus the protocol required some way to tell an engine that it's move is rejected and the game is over (it can cleanup or do what it has to do then)...
Taking a step away from gnugo itself, are these issues you want to concern yourself with?
Maybe, depending on whether I understand you correctly. In my opinion the arbiter has two reasonable responses for handling differing opinions on legal moves:
1. Stop the game and declare a loss for the program breaking the rules. (Either by determining this itself or asking a human to investigate.) 2. Stop the game, let a human operator fix the bug causing an illegal move to be made, and restart the game.
If an engine does need help identifying legal moves, this can be done by letting it open a GTP connection to a program with this capability, which it can ask about move legality. This would require some command like is_legal or all_legal, which are available in GNU Go. Probably these commands should be standardized but not required from every engine.
Also a question: does the protocol include the negotiation of these terms with the engine (ie a request for it's capabilities in the first place, and an ack or something implying it can play to the terms requested) (sorry if this should be obvious in the documentation, but i've not really gone through it...)
Again I'm not fully certain I understand the question, but one of the basics of the protocol is that a command from the arbiter is answered either on the form "=[id] result" or "?[id] error_message", where the former case indicates success and the latter case failure.
The only exact specification of results and error messages is currently the GNU Go source code. This needs of course be addressed by the standard. I'll try to specify the syntax and semantics of all the basic commands as they are understood by GNU Go in a later message.
issues like cr & crlf can be finessed by requiring an end statement, such as semi-colon, so that cr & crlf can be ignored. but then it isn't quite so elegant. but then what is the goal of the text: ie it seems that you want some sort of readability in the protocol, but why?
Primarily for two reasons: 1. Make it easy to debug the protocol, e.g. by inspecting a log to see what part failed to follow the protocol if the communication breaks down. 2. GTP commands can be written as scripts for regression testing, which was the first application of GTP in GNU Go. These test suites are more easily maintained if the protocol is readable.
/Gunnar _______________________________________________ gtp mailing list gtp@lists.lysator.liu.se http://lists.lysator.liu.se/mailman/listinfo/gtp
Just to clarify, when I said "Technically we could get by with 5 commands" I meant that this is the minimal thing needed to just play a game, not to give us an adequate starting set.
Don
I don't think this conflicts with the goal of simplicity, because I think there are very few core things needed. Technically, we could probably get by with 5 commands, but we might be far happier to include another 6 or 7 that will be incredibly useful later.
Also a question: does the protocol include the negotiation of these terms with the engine (ie a request for it's capabilities in the first place, and an ack or something implying it can play to the terms requested) (sorry if this should be obvious in the documentation, but i've not really gone through it...) jeff
My sense of things is that if there is any kind negotiation, it need not be "built in" to the protocol, in other words it's not something that engine has to know about. A smart GTP interface should be able to direct the negotiation itself with very simple commands and it would be soley up to the implementor to provide cool features like this.
I can imagine a cool arbiter program attempting to play some given ruleset, then having one or both engines saying "not_implemented" and then having the engine move on to another ruleset until it finds something both can play. Presumably the end user can have full control over when and how this happens.
Don
Phil wrote:
So far the minimal GTP command set includes:
Boardsize Komi Fixed_Handicap White Black Genmove_Black Genmove_White Undo (* provisional for now) Quit
Please use only lower case for the commands.
What else?
I think name, version, and protocol_version should be mandatory. They are all utterly trivial to implement anyway.
I also think we should have a specific command to clear the board. In the GNU Go implementation this can currently only (*) be achieved as a side effect of the boardsize command, but I don't think that's entirely intuitive.
Arguably there should also be commands for specifying rulesets. For the time being I propose that this is assumed to be set up through external channels (by GUI, command line options, or compile time options). Of course such commands will be needed for completeness but I think they should wait until the second revision of the protocol in order to keep the first revision really simple and to give us more time to decide how those commands should be designed. It's not at all clear to me what the best way to specify rules is.
/Gunnar
(*) Not entirely true, loadsgf can do the trick too.
The engines that connect to opengo require what you have below plus: chinese or japanese rules regarding suicide type of ko (superko or simple ko)
the referee require the above plus: how many games do you wish to play items such as how to record the game (sgf, ishi, etc)
engines themselves may require other options which are usually quite idiosyncratic and we don't touch that: logging data files style of play etc.
--- also, as gunnar has noted, the transmission protocol itself deals with error correction
--- issues like cr & crlf can be finessed by requiring an end statement, such as semi-colon, so that cr & crlf can be ignored. but then it isn't quite so elegant. but then what is the goal of the text: ie it seems that you want some sort of readability in the protocol, but why?
jeff
Phil wrote:
So far the minimal GTP command set includes: Boardsize Komi Fixed_Handicap White Black Genmove_Black Genmove_White Undo (* provisional for now) Quit What else?
So far the minimal GTP command set includes:
Boardsize Komi Fixed_Handicap White Black Genmove_Black Genmove_White Undo (* provisional for now) Quit
What else?=20
Shouldn't we start by defining what the objective of a "minimal" command set is?
We have 2 basic ways to go:
1. A purposely small, but reasonably competent set.
2. The very minimal needed to let 2 programs play a game, if you set up the engines correctly.
In the first option, we try to make a set that is "complete" in some sense. A good interface should have enough "interface" to be able to set up the engine. I should be able to tell my engine to use GTP and not have to worry about anything else unless it is highly engine specific.
The second way to go has the appeal that if you set up the engines with the right options beforehand, you can at least play a game and all programmers won't have to work too hard to implement this set.
Although this appealed to me at first, I have this feeling that whatever we do on the first pass, will end up being the defacto standard and nothing beyond this will likely get implemented. We will live with it for years! Maybe we should make some effort to make it complete enough to serve us well?
What I had in mind was a command to tell the engine what kind of ko rule is in use, a command to set the level (otherwise we have to build interfaces that don't have this ability) and a command to set up a board position. We also need a little bit about ruleset usage, because Tromp/Talor or Australian rules actually affect the final score if a program doesn't know these rules are being played.
True, we don't need these features, you can set up the engine ahead of time, but you cannot build a high quality interface this way, can you?
I'm not suggesting we add every bell and whistle, we still keep it very simple, we just add a couple of primitives that are enough to build a full blown interface with a good feature set, like loading and storing games and positions, setting a level, taking back moves, etc.
Is this too ambitious?
Boardsize Komi Fixed_Handicap White Black Genmove_Black Genmove_White Undo (* provisional for now) Quit
So what if we add something like this? (I need feedback here)
ko setup ruleset set_level version
This is short and sweet, and adds a lot of power to interface builders. I would rather add these and then say "it's ok to return "not_implemented" than to leave most everything out.
What do you think? By the way, I'm not sure what arguments to give to these commands, I'm very new to GO and didn't even know the rules when I started writing my program (and probably still don't.) How many KO rules are there? What is most common? Setup should take a list of white and black stone locations, without any kind of move validation expected. I'm not sure what levels to support, what is common usage in tournamants?
Don
P.S. So maybe we should call this the "recommend" gtp command set, and let programmers decide what to implement? Then we don't have to argue about whether "undo" is a useful feature or not.
On Wed, Aug 08, 2001 at 11:58:10PM -0400, Don Dailey wrote:
P.S. So maybe we should call this the "recommend" gtp command set, and let programmers decide what to implement? Then we don't have to argue about whether "undo" is a useful feature or not.
my $.015:
maybe a multi-tier approach:
perhaps have the minimal core be "required", maybe another set of commands be "recommended" and/or "optional" for the mostly solid stuff. And perhaps "development" or "alpha" for the totally changeable stuff.
--
Yes, that sounds reasonable to me. Really, the only difference in required and recommended is whether the end user sends back "not_implemented" or implements them. So we would provide a complete set, and then specify which ones absolutely must function to not break interfaces.
Don
From: leary@nwlink.com
On Wed, Aug 08, 2001 at 11:58:10PM -0400, Don Dailey wrote:
P.S. So maybe we should call this the "recommend" gtp command set, and let programmers decide what to implement? Then we don't have to argue about whether "undo" is a useful feature or not.
my $.015:
maybe a multi-tier approach:
perhaps have the minimal core be "required", maybe another set of commands be "recommended" and/or "optional" for the mostly solid stuff. And perhaps "development" or "alpha" for the totally changeable stuff.
-- _______________________________________________ gtp mailing list gtp@lists.lysator.liu.se http://lists.lysator.liu.se/mailman/listinfo/gtp
Although this appealed to me at first, I have this feeling that whatever we do on the first pass, will end up being the defacto standard and nothing beyond this will likely get implemented. We will live with it for years! Maybe we should make some effort to make it complete enough to serve us well?
I agree. This is the time to get it right. The command set needs to be sufficiently complete to allow sophisticated arbiter programs. If the command set is too minimal and e.g. requires manual setup of the programs between games, we've lost a lot of the advantages the GTP could offer over GMP.
So what if we add something like this? (I need feedback here)
ko setup ruleset set_level version
[...]
Don
This is a good start. A comment to set_level: SmartGo doesn't use levels, it uses the time available to determine how much time to spend on a move. So for generating a move, SmartGo needs to know whether this is part of a game being played (and how much time is left in the game), or whether it's just an individual move to generate within a given amount of time, such as 20 seconds.
One thing that has not been discussed much so far is the state we expect each program to maintain. Once we define the minimal state a Go program has to maintain, it may be easier to identify the appropriate commands, and it may also help define each command more precisely in terms of how it affects that state.
At a minimum, I'd expect each Go program talking GTP to know the following: (A) Board size. (B) Komi. (C) Handicap (fixed handicap, or remaining moves to play using arbitrary placement of handicap stones). (D) The ko and scoring rules in effect. (E) Board position. (F) Whose turn it is to play. (G) Enough knowledge of the history of the game to identify ko and repeated positions.
(A)-(D) get set up at the beginning of the game, while (E)-(G) change with each move played.
For each move to be played, the program needs to be told: (H) Time left (if this is a game rather than a problem position), or time to use for generating this move.
In addition, the program must be told (or have a way to request) the following information: (I) The time and overtime rules in effect (e.g. need to complete the first 250 moves within one hour). (J) If playing a game, the name of the opponent program. Many programs may ignore this information and will just play as fast as possible and not make any distinction between opponents, but if GTP is used to fully automate computer Go tournaments, I think this is needed.
Comments?
Anders Kierulf www.smartgo.com
Comments?
Yes, I think we are getting somewhere now. The programs do need a way to understand what the level is, even if they don't really use that information, we can't assume this. I'm not familiar with tournament practice and so will need a little help.
What time control conditions are in widespread use? Is it similar to tourament chess? ie:
. Game in x minutes. . n moves in x minutes
and in programs we might see levels like:
. Average n seconds per move.
Some older programs have something as simple as level 1, 2, 3 etc without this being really defined in terms of playing time, perhaps like gnugo's levels of 1-10.
One thing about gtp as I understand it, it seems to be command driven, an interface gives commands and gets back a response. There is no provision for the engine to initiate communication beyond this, unless we do it on another channel. I'm not sure we really need this, but it might be useful (while more complex.)
One thing I would like to do, is to command the engine to stop searching and play immediately. This is useful for long thinks, if your engine is capable of thinking for long periods of time. But if the interface is waiting for a move to come back, is it forbidden to send a 'move_now' (or even 'quit') command? Things get out of sync, especially if you do not use the id feature of GTP.
Don
From: "Anders Kierulf" anders@smartgo.com
Although this appealed to me at first, I have this feeling that whatever we do on the first pass, will end up being the defacto standard and nothing beyond this will likely get implemented. We will live with it for years! Maybe we should make some effort to make it complete enough to serve us well?
I agree. This is the time to get it right. The command set needs to be sufficiently complete to allow sophisticated arbiter programs. If the command set is too minimal and e.g. requires manual setup of the programs between games, we've lost a lot of the advantages the GTP could offer over GMP.
So what if we add something like this? (I need feedback here)
ko setup ruleset set_level version
[...]
Don
This is a good start. A comment to set_level: SmartGo doesn't use levels, it uses the time available to determine how much time to spend on a move. So for generating a move, SmartGo needs to know whether this is part of a game being played (and how much time is left in the game), or whether it's just an individual move to generate within a given amount of time, such as 20 seconds.
One thing that has not been discussed much so far is the state we expect each program to maintain. Once we define the minimal state a Go program has to maintain, it may be easier to identify the appropriate commands, and it may also help define each command more precisely in terms of how it affects that state.
At a minimum, I'd expect each Go program talking GTP to know the following: (A) Board size. (B) Komi. (C) Handicap (fixed handicap, or remaining moves to play using arbitrary placement of handicap stones). (D) The ko and scoring rules in effect. (E) Board position. (F) Whose turn it is to play. (G) Enough knowledge of the history of the game to identify ko and repeated positions.
(A)-(D) get set up at the beginning of the game, while (E)-(G) change with each move played.
For each move to be played, the program needs to be told: (H) Time left (if this is a game rather than a problem position), or time to use for generating this move.
In addition, the program must be told (or have a way to request) the following information: (I) The time and overtime rules in effect (e.g. need to complete the first 250 moves within one hour). (J) If playing a game, the name of the opponent program. Many programs may ignore this information and will just play as fast as possible and not make any distinction between opponents, but if GTP is used to fully automate computer Go tournaments, I think this is needed.
Comments?
Anders Kierulf www.smartgo.com
_______________________________________________ gtp mailing list gtp@lists.lysator.liu.se http://lists.lysator.liu.se/mailman/listinfo/gtp
On Thu, Aug 09, 2001 at 07:13:23PM -0400, Don Dailey wrote:
What time control conditions are in widespread use? Is it similar to tourament chess? ie:
A decent summary of time systems: http://kgs.kiseido.com/en_US/help/timeSystems.html
--
-
Anders Kierulf -
Don Dailey -
Gunnar Farneb�ck -
jeffg -
leary＠nwlink.com -
Phil