Comments/Ratings for a Single Item

That is purely a Fairy-Max problem. It has a pretty sloppy CECP parser, which doesn't bother to recognize most commands that require no processing (e.g. accepted, rejected, computer). 'Result' is also a no-op for an engine that doesn't implement learning. A GUI should not worry if an engine complains against non-essential commands. I think WinBoard only reacts to rejection of time/otim commands (concluing that it must be ealing with GNU Chess, and refraining from sending these commands for the rest of the game), and 'analyze' (informing the user that the engine doesn't support it, for the benefit of WB v1 engines that haven't announced this in advance through a feature analyze=0).

Ok, good to know that I don't need to worry about that.  I was wondering because I have been experiencing some stalls when running ChessV vs. Fairy-Max games in batch mode and it only happens when Fairy-Max declares a draw and then only sometimes.  I don't think it's related to this issue though.  So I'll have to keep digging.

Regarding playing CwDA through CECP, this game presents some interesting challenges, most notably army selection.  Given the current state of affairs, there are two obvious approaches, but neither is great...

1.  We could consider each army match-up to be a different variant - FIDEs vs. Rookies is one variant and Rookes vs. FIDEs is another.  This is a little cheesy and leads to variant explosion.  Essentially CwDA becomes two to the power of the number of armies different variants.  That said, this is probably the easiest.  I think ChessV can do this now with no modifications to the code at all.  We could always start here and be up-and-running in no time and just back out these "dummy" variants later.

2.  We can do what you are doing now - have engine paramters for army selection.  This is arguably better, but I don't love this option either.  We are adding engine-level settings for things that only apply to one game.  This is somewhat cheesy, unintuitive, and doesn't scale well.  What if we have a different variant that also has army selection but different armies for choices?  What happens when we start adding configuration parameters to other variants?  (ChessV already has configurable game-specific options for other things, such as the castling rule in effect in Capablanca variants.)  I don't mind altering ChessV but when I add things I always try to add them in a general way.  I'd rather not add code just to "hack" this one specific variant.

I think what we want - ultimately - is a new concept.  ChessV has "game variables" which are configuration options that apply only to specific games.  Conceptually, this is exactly what we are trying to accomplish.  Ideally, CECP would be given a new feature which engines could declare support for.  If an engine announces support for game variables, the GUI could then ask it, for any given game, what options it supports.  But I realize that this requires significant changes to XBoard/WinBoard, and it is probably no small undertaking.  But I think it's something worth striving for in the long term, and would pay dividends down the road.

Actually, we might be able to get there with the first step just being naming convention.  If we make "cwda" to be the Chess with Different Armies variant, maybe the options could be named like "@cwda:White Army", for example.  At first, WinBoard could just treat them like any engine options.  But in future, it might parse the option name to understand that it only applies to variant "cwda".  Just thinking out loud here ... haven't considered it deeply.

Fairy-Max currently also uses approach (1), which doesn't require engine or GUI to be aware that there is some relation between the 'sub-variants'. It uses the general 'engine-defined variant' mechanism for this, where it can just make up any name it wants and tell to the GUI that it plays it, after which the GUI makes it available for selection by the user. And when it gets selected the engine sends details like board size, pieces and moves to the GUI. Before that it did something similar by using 'fairy' as wildcard variant, where an engine-defined option then could select what 'fairy' actually meant (and a similar flow of rule info to the GUI when it got selected). Like you, I am not very happy with the resulting variant explosion. (Although this is not so bad as you say, just N^2 instead of 2^N.) Even with the 4 'standard' armies it overflows WinBoard's New Variant dialog with 'FIDE-Nutters' and similar variants. So I put them at the end, and kept in the old selection mechanism through an option.

So for the new engine I try to find something better. For an engine dedicated to CwDA the per-player option for army selection is a logical choice. But, like you say, if it really is a general multi-variant engine, it could lead to an explosion of options meaningful in only a single variant. I also found it quite annoying in practice that I have to set the option on both engines, when doing engine-engine games.

An explicit method for defining variant-dependent options like you propose would be a possible solution. An even more general case would be where options presented to you by the GUI would depend on the setting of other options. Processing the option definitions that the engine sends at startup in XBoard was not limited to the handshake phase, but originally it was not possibe to redefine or remove an option once it was defined, as newly received option features were always appended to the option list. But to make this behavior useful I made it such that receiving 'feature done=0' clears the entire list of options for that engine in the GUI. So an engine can redefine its options at any time. The application I had in mind was that engines could send a minimal set of options by default, including a button 'Advanced', and when receiving the activation signal for that option, resend a more extensive list of options (perhaps including a 'Minimal' button to switch back to the minimal set). Or even have several 'submenus' selectable by buttons from the base dialog (e.g. Eval Parameters, Search Options, EGT Options) each with a button in them to return to the basic list. Other applications would be 'collective' options that alter the setting of other options (e.g. 'Restore defaults'); the engine could then resend the option list not with different option names, but with different defaults, to make the GUI aware of the new settings. Main problem with this in the GUI is that the settings dialog gets redefined while it is open in such cases, which I think I tentatively solved by simply closing it (not sure anymore). If a change of option list occurs due to selection of another variant, that problem would not exist, as the settings dialog would be closed at that time.

Now from a protocol point of view this is a rather inelegant solution. But it is very general, and it works. A multi-variant engine at startup can only declare its general options (e.g. 'Resign Threshold'), but on reception of a 'variant' command resend those (starting with done=0) plus all options specific to the specified variant. If the user would then open the settings dialog he would see the new options. The engine would have to be smart enough to suppress resending of the option list if the variant did not really change, though, to prevent that clearing the list would make the GUI forget the previous setting of these options. Or be smart enough to make the previous setting the new default when it resends the list. (Not entirely trivial, as the 'new' command formally switches you back to 'variant normal' at the start of every game. So it should remember the argument of the latest 'variant' command, and on reception of a new variant command compare with that, and redeclare options only if there is a change that would lead to a change in options.)

It also seems CECP is in need of a mechanism for engine-defined options that work on both engines. Normally the corresponding features would be controlled by standard options, which might even use a different syntax (like 'feature memory=1' to set the hash size). But for features the GUI is oblivious of this cannot be used. But if the engine could define an option as 'slave', the GUI could send any change of the value to the other engine as well (provided both engines declared it). You could then change their value in the dialog of either engine, while the GUI would guard the consistency of the settings.

Sorry it has taken so long to get back to you on this.  A lot to chew on here.

First you mentioned engine-defined variants the GUI doesn't understand but still allows to be played.  ChessV doesn't do this, and it's not high on my list of things to do, but there is certainly some value to it so it would be best if whatever we settle on supports this.

Then you mention my game variables concept and the more general concept of allowing engines to suppy additional options as things change (variant selected, other options selected, option buttons clicked ...)  Certainly the more general concept has value, but I also think there is value in being able to specify that an option is a game-specific variable.  This would make clear that, not only does the option only apply to a specific game, but that the option affects the rules of the variant being played and not just the internals of the engine AI (although it could certainly do that too.)  This tells the GUI that, in order to play, both engines must support the same options, and both should be automatically set to the same values after the user selects them.  And I think it makes sense that these options should be presented differently because they are an essential part of selecting what game is being played.  In ChessV when you select Chess with Different Armies, the first thing you get is a dialog box to pick your armies with random armies pre-selected.  (The same thing happens if you pick Fischer Random Chess - you get a dialog with a random position number selected, with a preview, but you can change it if you want to play a specific position.)  I think game variables can offer a better user experience - if you don't care about, or even don't understand hashtable size, etc, who cares, but you better understand what the armies are.  Picking the armies is part and parcel of picking what variant you want to play.  And, also, some engines might play CwDA and offer extra armies that others do not.

Another thing I like about game variables it the potential they offer to allow an engine to play variants it wasn't specifically designed for.  ChessV has some understanding of this now.  If the user selects Corridor Chess, it is smart enough to offer as options any engine that supports both orthodox chess and the setboard option.  Likewise, if it offers variant capablanca and has setboard, it will be offered as an option for Modern Carrera, but not for Schoolbook because of the flexible castling.  (You did propose a clever workaround for this but I haven't got around to implementing it.)  But engines that support flexible castling should offer the "Castling" option of "Flexible" in variant capablanca.  Then, even if they don't know Schoolbook, Grotesque, or Ladorean, they can still play them, along with new variants that meet the same rules.

10x8 is popular, in the sense that there are lots and lots of variants, but almost all use one of a few castling rules, which I have named "None", "Standard", "Long", "Fexible", "Close-Rook", and "CRC".  Likewise, on a 10x10, there are only a handful of castling rules, and a handful of rules for the pawn's multi-step moves that cover most games.

About a year ago you mentioned working towards a standard for defining variants so you don't have to configure Fairy-Max in one way and SjaakII in another.  I think this is also worth pursuing - it would be awesome if we could acheive it, but this is a steep mountain.  I respectfully suggest that this would make a good first step.  This is the low-hanging fruit...

The castling issue is a good example of the different design philosophies that are possible, so let me respond to that first. If an engine supports Capablanca Chess, it is unlikely to support flexible castling. The only reason why the author would care to support such castling is that he is aware of Scholbook Chess, and wants the engine to be able to play that too. But if he wants that, why would he want the user to alter the start position (through setboard) and the castling rules (through an option) separately? It would be much simpler, for him as well as the user, to just add Schoolbook as an additional variant the engine can play. If there are other 10x8 variants that use flexible castling, but have a different start position from Schoolbook, these could then be played as sub-variants of Schoolbook (through setboard). Variants without castling are sub-variants anyway, as you can simply specify a start position without castling rights.

Having a castling option is a step towards defining the variant for the engine through option settings. If you have an option for castling, then why not one for how the super-piece moves (Queen / Chancellor / Archbishop)? Or options for how any piece moves? Or how large the promotion zone is? Or what you can promote to? (To play Janus as sub-variant of Capablanca.) This would all be possible, of course; it would converge on a design where the GUI manages all game descriptions (be it hard-coded or through a configuration file that the user can edit), and then instructs an engine how to play it.

This is not how existing multi-variant engines such as Fairy-Max, Sjaak and Nebiyu work, however. These manage the file with game definitions themselves. Which seems the logical way to do it, as then the file can also contain engine-specific info. (Like which pieces have mating potential, value of pair bonuses etc.)

Because of the way XBoard started (as merely a graphical front-end for an already complete ASCII-based user interface) things tend to be 'engine-centered' in WB protocol. From the early days on XBoard could already be used to play variants (as long as they were 8x8 with no more than 6 piece types, such as Shatranj) by switching legality testing off. (In fact in the earliest version it could not even be switched on, as legality testing was not implemented at all. Even when I first started working on WinBoard myself in 2008 it did not know about castling rights, and would always accept any castling.) Historically it was up to the engine to refuse moves it considered illegal, on which the GUI would retract the move and relay the info to the user. Downsite of this is that the GUI cannot indicate target squares of a move when pieces might move different from normal, and that it will generate crappy SAN (with missing or redundant disambiguation, but which it would still understand itself on reading it back).

I generalized the 'legality off' capabilities a bit by attaching a meaning to some otherwise unused combinations of mouse clicks; e.g. any sideway move of a King by more than 2 spaces is now interpreted as an attempt to castle with the corner piece in that direction (whatever it is), while dragging the King on top of the corner piece would be castling with it where the King steps one. (In all cases the corner piece ends up on the other side of the King.) Unless Fischer castling is specified by a GUI option (which by default is set for FRC and CRC), then capturing an own backrank piece with the King indicates castling with that. This covered all cases I was aware of, so the GUI doesn't really have to know the game rules to interpret the user move; assuming that what the user tries is legal, it will interpret it correctly. The engine watches over the latter, by forcing the GUI to take back whatever illegal thing the user might have done.

For better GUI performance w.r.t. SAN and target-square highlighting, however, it is essential that the GUI is also aware of the rules. Logically there are 3 ways to accomplish this:

1. Both the GUI and the engine are programmed / configured for the same rules independently.
2. The Engine transfers the rules to the GUI
3. The GUI transfers the rules to the engine

Your castling option is an example of the latter; through engine-defined options any info can in principle be transferred in this direction, but some standardization would have to be developed to make it possible for the GUI to do this automatically (rather than having the user decide on a per-engine basis what is necessary). For the start position the protocol has the setboard (or edit) command. For case (2) I extended the protocol with a 'setup' command as a reverse setboard, (plus a list of participating pieces) and 'piece' commands for defining piece moves.

Fairy-Max and Sjaak || use method (2); their game-definition file specifies how castling is to be performed, and (if this is non-standard) they inform the GUI of it as part of a King-move definition, e.g. "piece K KisO4", where the isO4 is the XBetza notation of a castling where the King moves 4 squares. Through this info XBoard itself can then already refuse any attempt of the user to move the King other than 1 step or slide it 4 (if still virgin etc.) without first consulting the engine. (And does this by not even highlighting other King destinations, and in general refusing any move to a non-highlighted square.)

I guess we agree that method (1) is undesirable; this was basically the situation that existed before introduction of the setup and piece commands, where the user had to provide a FEN (and even in an indirect way, as a .fen file with a GUI option referring to it) and a -pieceToCharTable option to specify the FEN's meaning, and switching off legality testing when standard glyphs were used to represent exotic pieces not in the standard set. For the case of castling it seems to me that method (2) is more direct than (3). In all cases the engine would have to announce something to the GUI at startup, either that it has a castling option that features 'flexible' as a choice, or just that it supports Schoolbook. In the case of the option the GUI will have to recognize it, look up in a list of known variants which variants would need any of the available settings for that option, and enable selection of those variants by the user (if their other requirements are also satisfied). In the other case it can just enable selection of 'Schoolbook' without having to worry about anything, or even know what 'Schoolbook' means. When the user decides to select Schoolbook, in the case of the option the GUI would now have to set the castling option in the engine to the value it knew to be required for Schoolbook, as well as enable that type of castling in its own move generator used for legality testing / square highlighting / SAN parsing and generation. In method (2) it would just configure its move generator likewise because the engine tells it so through a piece command. Again, it did not need to have any pre-programmed knowledge for that. The 'piece' command can be considered in this case just as a (somewhat cryptic) command to set a GUI option for the type of castling. This is much less of a hassle for the GUI.

I am perfectly happy with XBoard being a 'dumb' GUI, which leaves the engine in charge. But if you want ChessV to be a GUI that knows about many variants, and makes conclusions about which variants not explicitly supported by the engine can still be played as sub-variant of some other, supported variant, method (2) would still allow you to do it. In all cases the GUI would look which variants the engine supports, and what aspects of variants the engine allows to be changed in general (startposition by setboard, possibly other options), and then can run through its list of known variants to see which can be run as sub-variant. It doesn't really matter if a 10x8 variant has to be run as Capablanca + setboard + castling=flexible or through Schoolbook + setboard. If you want the GUI to have intrinsic knowledge of what Schoolbook is, it can simply ignore setup or piece commands the engine sends to specify start position and castling type.

As to the selection of armies:

We definitely agree that it is useful to have options that are common for both engines. Putting some common prefix, and perhaps an agreed-upon special sysmbol in front of a set of option names is always possible, and doesn't interfere with any interface that would not treat it as special.

The main thing I worry about is that army selection in CwDA, and even more clearly selection of the 'guest pieces' and where they have to go (as replacement or through gating) in Superchess and Musketeer Chess, setup of the super-pieces in Metamachy, are all really parts of game play. According to the official rules of those games there is a prescription for how the players negociate an initial position. In the XBoard implementation of Superchess I dodged that problem by implementing it as a shuffle game; my justification for that was that this 'game prelude' is really a work-around for human-human games, where there is no third party that could make the decision for them in an unbiased way. When a computer is involved, it might as well make the decision for them. If they don't like the proposed setup for a human-comp game they can just keep pressing 'New Game' until they do. If comp-comp games need a specific start position you just play them as games from a setup position, and provide the desired FEN to the GUI that runs those games. Jocly, OTOH, takes these game preludes seriously, which makes the implementation of the mentioned variants enormously more complex, involving a lot of non-standard code. (Where other variants are implemented by simply giving a list of piece properties and their start location.)

For CwDA the issue is a bit more subtle, as there are potentially so many armies (even the Wikipedia lists many armies I had never heard of) that it is impossible to give the pieces a unique 1-letter ID. So which army the engine has to play with cannot be unambiguously specified through the FEN in a setboard command, even if the GUI would make the army selection at random.

Historically WB protocol contained a feature that would allow user-engine dialogs as needed in a prelude: the askuser command. This is equivalent to a single string option: in response XBoard will pop up a dialog where the user has to enter a text, which will then be sent to the engine on 'OK'. In the latest XBoard I tried to expand the possibilities by allowing the engine to request opening of its entire settings dialog (as defined at that time) 'spontaneously'. So it could redefine its option list to define a dialog for the purpose, starting with feature done=0 (to temporarily remove its normal options), and end with done=2 (the forced popup request). This could have two comboboxes in it for the available armies, or (for Musketeer chess) comboboxes for selecting piece types and the files at which they would gate, etc. Problem is that such things would not work in automated engine-engine games unless the GUI itself would be able to set the presented options. But in WB protocol the engine can know if the opponent is a computer, and then refrain from requesting the popup, and use the default values, or the values previously given for those options.

Things are really ugly if the rules for the prelude require the players to take turns on selecting things; in a human-comp game you would then have to go through a sequence of dialogs where the engine each time says "I did this, now what is your next step?", before we can actually start moving pieces. And what if it makes sense for the engine to really think hard about its decision? Should its clock be running?

I got the impression that you would like to have a separate dialog for setting the variant-specific options of the engines, which perhaps pops up automatically after you selected the corresponding variant. This would be feasible for CwDA, but in view of the above I wonder if it would be sufficient in general.

Regarding army selection, and other setup options:

I got the impression that you would like to have a separate dialog for setting the variant-specific options of the engines, which perhaps pops up automatically after you selected the corresponding variant. This would be feasible for CwDA, but in view of the above I wonder if it would be sufficient in general.

Yes, this is what I do.  It populates the dialog with any game variables that are not yet specified.  There are reasons why they may already have values.  The game may provide values, in which case they are there just as a configuration options, but the user isn't normally prompted.  Or the user may have selected a game that is derived from the game which fills in the options. (You could make a new variant derived from CwDA where the armies are always both Nutty Knights plus whatever other changes.)  Or the options may already be specified because the user has opened a saved-game file, which of course must contain this information along with the game.

You raise a number of other things though which could certainly be considered related.  For games where players take turns making choices or placing pieces, I view this as part of the actual game.  Ideally, the GUI prompts you when it is your turn to make a decision.  And the computer's clock should certainlly be running while it is considering its own choices.  That said, I have not implemented anything like this yet.  I haven't implemented any of the games you mention...  And for some choices, it may not be obvious how they would be communicated in the protocol.  For placing pieces, though, the existing @-notation for dropping pieces seems like an obvious choice.  This does, of course, require a lot of new code...  I agree it could be considered an "ugly" problem from a code-writing perspective.

In the latest XBoard I tried to expand the possibilities by allowing the engine to request opening of its entire settings dialog (as defined at that time) 'spontaneously'. So it could redefine its option list to define a dialog for the purpose, starting with feature done=0 (to temporarily remove its normal options), and end with done=2 (the forced popup request). This could have two comboboxes in it for the available armies, or (for Musketeer chess) comboboxes for selecting piece types and the files at which they would gate, etc.

Having this come from the engine just does not make sense to me.  There are potentially two engines but only one GUI, so it seems to me the GUI should be "driving the bus" here.  Which engine makes the request?  What if the other engine doesn't support the same armies (or other options)?  You could just pick one engine and stipulate that to run a match between engines they must support the same things (or that the use will only pick options supported by both.)  But then the GUI needs to tell the second engine, so the options would need to be named the same... It seems this approach still has game variables, and the GUI still must have an understanding of this concept to facilitate the hand off.  I think it is more straight-forward for the GUI to know which game variables and associated options the engines support, (the engines announce this anyway), and present the user a choice of the options both support.

I'll hold off on addressing the other topic for the moment.  I don't have enough time at the moment to think through it all and draft an intelligent response.

Note that I don't want to argue that this is an ideal solution for everything; I just indicate what is already possible in XBoard. The possibility to pop up complex dialogs was inspired by Evert Glebbeek's remark that he would like an engine to be able to use the GUI for design of a new variant (rather than requiring the user to edit config files), through an interface similar to the Design Wizard I put on the Interactive Diagram article here. The described method was trivial to implement in the GUI, as everything was already there. So it was just a matter of allowing the engine to invoke it (rather than a menu click).

Some steps occurring in typical preludes could indeed be covered by regular moves, so that they would not need any special provisions at all on the GUI side. E.g. variants that start with setting up the army turnwise on an empty board could use drop moves, and the engine can simply refuse premature attempts to move around pieces in the setup phase. This can be extended by adopting the conventon that a drop move on a square that is occupied by a friendly piece indicates a swap, and takes the original occupant back into the hand. That way a Superchess prelude could be implemente as: white substitution 1, black substitution 2, obligatory white symmetrizing substitution 2, obligatory black symmetrizing substitution 1, white substitution 3, etc. In Musketeer Chess you don't have to pick just pieces, but also assign them to a file. (Very bad idea, btw...) This could be done by making extra (initially empty) board ranks to drop the selected pieces on. But this would then turn their later gating into a double move, where they have to step forward as a side effect of the piece in front of them moving away. Of course if the protocol extension is used that allows general multi-moving, (as in the WinBoard Alien Edition), this would be no problem. It does seem a bit unclean, though (and for clarity the extra ranks should be differently colored than the normal checkering, so a lot of GUI support). Some people would attach value to the piece and its gating location being visible at all times, but I don't consider that an absolute necessity. (There is also no optical clue for whether you are allowed to castle, so not all game state is always visible.)

I am dwelling on this to decide whether there is any market for a protocol extension that could exchange arbitrary messages between the players (whether engine or human), which would then also switch the clocks and appear as a move in the PGN. E.g. "pseudomove <string>" both for engine->GUI and GUI->engine, depending on who's turn it is. At first glance this seems of little use if the GUI does not know what <string> means. But it could be used to change hidden game state, for instance to assign a file to a piece in hand, to play Musketeer as a modified Seirawan Chess (where the GUI assumes you would always be allowed to gate anywhere). The engines could then communicate the allowed gatings to each other through the pseudomove command, send the actual gating in S-Chess format (e.g. f1g3c to gate a Cannon), which would be understood by an S-Chess-supporting GUI. GUIs especially designed for Musketeer could apply some visible clue to indicate the gating pieces and files, which could go as far as displaying a piece just beyond the board edge. But you would not be dependent on such a GUI feature for playing engine-engine or human-engine games. (If you are willing to put up with the hidden game state.) In the latter the GUI would have to present the <string> to the opponent, and allow the user to enter one by typing. (The general mechanism.)

Note that playing different engines against each other is really something that almost no one does. TalkChess is really a very unrepresentative environment in that respect, full of engine developers and testers. But engine developers, and especially variant-engine developers, are a rare breed. In XBoard the 'second engine' is only used in 'Two Machines' mode. But XBoard is very tightly coupled to the 'first engine', taking it along with everything you do (e.g. in Edit Game mode). This because it originally was dependent on that engine for legality checking of the entered user moves. So what XBoard does is ignore the second engine completely (it is not even started up) until the user presses 'Two Machines'. Then that engine is started, and if it then turns out to not support the current variant, the mode switch is refused, and an error popup appears reporting "Second engine does not support this variant". For variants that are not fully specified (such as shuffle games) the first engine is in charge, and any start position it specifies will be loaded through setboard into the second engine. The engines should better be compatible w.r.t. how the pieces are called and how they move, but that always applies. If they don't agree on the initial setup (like the probably would, in shuffle games) it is no problem, the position indicated by the first (if the GUI or user had no desires on this) would be force-fed to the second to setboard. If they would disagree about piece names or moves this would wreck things, and XBoard currently doesn't check this. But this would always be the case, even in standard variants known to the GUI, where there isn't anything that could be checked. It just means one of the engines is non-compliant.

I guess the real problem is that different CwDA flavors are in fact different variants, so that treating them as a single variant leads to all kinds of unnatural problems. In a single variant, only partially supporting it (like: "I can play Chess, but the Knight is not implemented") is not an option. But there is nothing wrong with supporting CwDA except the Nutters. The only real problem is the unmanageable proliferation of variants this causes when you support many armies. Perhaps the GUIs should just provide a smarter selection method for variants in general, not just allowing ticking of one radio button from a list of 'full' variant names, but allowing a variant name to be synthesized from two parts, each selectable by a radio-button from a list. E.g. the engine could specify it plays rookies~CWDA and CWDA~nutters, where the ~ in the variant name would be recognized by the GUI as having the special significance of defining a partial variant name, where the all-capitals part then would be considered a place holder as well as a group identification. It could then present lists of such 'left' and 'right' partial names in the New Variant dialog, each with their own group of radio buttons and allow selection of two compatible (i.e. belonging to the same group, here 'CWDA') partial variants (and refuse 'OK'ing the dialog for incompatible selections), and combine the partial names to 'rookies~nutters' to get the selected variant. Depending on the preference of the GUI developer this could also be two combo-boxes (for left and right partial name), or even an arbitrary number of pairs of comboboxes (one for each variant group). The list of 'full' names would probably list 'selected below' as one of the possible choices to enable the comboboxes for selecting the partial names. Or one radio button for each variant group. E.g.

o normal

o xiangqi

o shogi

o CWDA: [combobox1] ~ [combobox2]

[Cancel] [OK]

The (first) engine would have full control (through its variants feature) over how this New Variant dialog would look. (And pedantic GUIs could ignore things in the list that they don't like or don't know; e.g. the standard edition of WinBoard would ignore variant tenjiku.) If a match with the second engine is attempted, the GUI will check if the current variant is in that engine's variant feature directly, or as a combination of partials. And refuse the request to play the engines against each other when it isn't.

This moves the selection of armies from the engine's private option settings to the GUI's variant-selection dialog, which is what you want. The variant is a common setting for both engines, so it also removes the problem of  'engine coherence', which is what I like about it.

I have some misgivings about engine-defined options that should be standardized, recognized by the GUI, and displayed in a separate dialog. This is just not what the option feature was desigintended for. For comparison: WB protocol allows 'size overrides' on variant names, like 5x5+5_shogi. (And users can select such variants in XBoard by using size-override rank, file and holdings controls in the New Variant dialog.) Normally the engine would consider each differently overridden version of a variant a separate variant, and list it as such in its variants feature. (E.g. Shokidoki lists shogi, 5x5+5_shogi (= mini-Shogi) 6x6+6_shogi (= Judkin's Shogi), 7x7+6_shogi (= Tori Shogi) and 8x8+6_shogi (Euro-Shogi). But I also did define a standard pseudo-variant 'size', where 10x10+0_size in the variants feature would imply that the engine will accept size overrides for board sizes up to 10x10 (but no holdings) on any of the reported variants. This is very similar in purpose to your concept of game variables, like having spin options for ranks and files with maximum 10. The equivalent of your castling option in this form of the protocol would be to allow a 'flex_' prefix on any variant name to indicate that variant uses flexible castling, (e.g. flex_capablanca) and define a standard pseudo-variant flex_castling to indicate that every supported variant also is offered in the 'flex' flavor. (Basically just a short-cut, to prevent very long variant lists.) GUIs understanding the flex_ thing could display / enable a checkbox 'flexible castling' in their New Variant dialog. This way of doing things looks more natural to me, because it puts the modifiers where they really belong: they basically create different variants, which deserve to have different names.

After having slept on it I realize there is a problem. General configurable variant engines will usually be configurable to play CwDA, but they will have been designed without the peculiarity of army selection in mind, and thus have no special features for it. The only way for such engines to play CwDA is by adding each flavor as a separate variant. If a GUI would not support this method, it would thus become incompatable with most engines that can play CwDA, as there will be very few dedicated CwDA engines. The partial naming I proposed in my previous posting requires engine support that they would not provide.

So it seems impossible to prevent variant proliferation in the engine's variant feature of WB protocol. But that shouldn't stop a GUI from keeping this proliferation out of its variant-selection dialog. So I want to propose a naming convention that would give special treatment to variant names of the form A~B(C) (e.g. FIDE~Nutters(CwDA) ). These should be considered as belonging to the "variant group" C, which should be enabled on par with primary variants. The GUI would collect lists of all A and all B occurring with that C in the engine's variants feature, and offer a separate selection method (e.g. a combobox) to select an A and a B.

This would give rise to rather long variant names, which would be clipped in the display of some existing GUIs; this is why I put the least-informative part at the end, rather than having something like CwDA:FIDE~Nutters. In addition, GUIs that do not support this convention would display a very long variant list, which they might also truncate. (This problem exists already with WinBoard + Sjaak II without the help of CwDA, and the latter provides a work-around for it through an engine-defined option defining the meaning of wild-card variant 'fairy'. In engine-engine tourneys selection through a WinBoard command-line parameter is always possible, though.) But newer GUIs would reduce the entire group to just a single item in their primary selection menu no matter how many flavors the engine reports, and provide additional methods for sub-selection of a flavor. (E.g. comboboxes behind the radio button that selects the item, or a separate dialog that pops up after the item gets selected through the primary dialog.) Engines don't have to know anything about this 'streamlining' of the variant selection.

P.S. due to construction of a new kitchen in my home I probably won't have much opportunity to think about these things or post here in the upcoming week.

I think this is a reasonable proposal.  For games with required parameters, I can provide some information about how the game variables are mapped into the xboard variant name.

It seems we will need two-letter piece names though.  One thing I do not want is for the notation of a piece to change depending on what the opposing army is.

I hope your kitchen renovation has gone well!

The problem with multi-letter ID is that they do not work in FEN when mixed with single letter. (And even if not mixed, it leads to FENs that are unreadable for humans). As an alterative to FEN there could be DEMN, which uses piece IDs whith a first-letter upper case and the optional following letters in lower case, and then lists the white and black pieces in separate sections concatenated by #. (Advantage is that that this also works for multi-player variants.) I was still not happy with the human redability, though.

In XBoard I finally adopted the solution of ' dressed letters', where each piece ID is a single letter (captital / lower case used to distinguish color in FEN), optionally followed by a punctuation sign, like quotes or an exclamation point. So L, L' , L`, L" and L! all indicate different pieces. This gives FENs that are quite readable. It can also be use in SAN, but there alternatively only the letter could be used, conventional coordinate disambiguation sloving the problem that G can both be Gold an Great General. (The WinBoard Alien Addition currently uses this method in Tenjiku Shogi.)

Note that there is not really any need for pieces of opposing armies to have different ID; it should always be clear whose turn it is (in SAN), or which color (in FEN). So both Bishop and Bede can be B without causing any problems, as they will never occur in the same army.

Note that there is not really any need for pieces of opposing armies to have different ID

Actually, you can't count on that.  A pawn may promote to any piece from either army.

XBoard I finally adopted the solution of  dressed letters

Yup.  I've already added support for this.  You made a good case why this was logical for shogi variants with lots of pieces.

The problem with multi-letter ID is that they do not work in FEN when mixed with single letter. (And even if not mixed, it leads to FENs that are unreadable for humans).

I use an underscore as a prefix to force recognition of two-letter notations.  It is not required if the first letter of the two-letter code does not represent a valid piece on its own, although you can use it anyway.  The underscore is never displayed to the user.

As for human readability, I don't really understand this concern.  They're not readable already.  Can you look at a chess FEN and visualize the board (like which pieces are on the same ranks or diagonals?)  I certainly can't.  To me FEN is just a dense encoding for computer interpretation.  In ChessV, you can just paste an FEN into a dialog and it will load it.

Actually, you can't count on that.  A pawn may promote to any piece from either army.

O sh*t, I hadn't thought of that. Probably because it cannot happen in Fairy-Max.  I still wouldn't be unhappy with the dressed-letter approach, though. Charging R and N could be R' and N', Bede could be B'. To me that doesn't look any worse than CN, RN and Bd. For the classical armies this would give: NBRQ, WFB'A, F'N'R'C, W'HSM. In almost any situation the punctuation would be irrelevant, as you would virtually never promote to Waffle, Fibnif, Woody Rook etc. So if we choose to use conventional disambiguation in SAN rather than punctuation, it would never virtually never need to disambiguate anything. And if the FEN is considered just an arbitrary string for computer consumption, it becomes irrelevant how it is done there anyway.

I use an underscore as a prefix to force recognition of two-letter notations.

I did something similar for the internal representation of start positions in my engine HaChu, using colon instead of underscore, but I wasn't very happy with the result. If auxilary symbols are used, you might as well just require all pieces to be separated by commas. (This is the notation that is also used in the TSA rule books for the various Shogi variants.) Forsyth notation was originally conceived for human reading, in newspapers. For computers it would be much more convenient to have some fixed-length format. You either don't care much about the length, (like in GUI-engine communication), or you care so much (in database applications) that FEN is hopelessly inefficient size-wise.

My problem has always been that XBoard would have to be changed in uncountable places to support even just 2-letter IDs. Which makes me inclined to consider options that objectively might not be the best. With the dressed letters the Id currently still fits in a byte, and only on reading or writing a mapping has to take place of the ASCII range 64-128 on the three other available 64-bit ranges. If I would manage to change all variables ever used to hold piece IDs to ints, I could use a similar strategy in storing mult-letter IDs e.g. as firstLetter + 256*secondLetter + ...

This night it occurred to me that for CwDA we could adopt the convention that L' means a piece that is normally in the opponent army. Then B can be Bishop for FIDE and Bede for Clobberers, and in the extremely unlikely case FIDE would want to promote to a Bede, this would become B'.

On vacation with only a phone so I can't type a long response.  I'm sorry, but I am not in favor of this idea.  ChessV expects pieces to have the same notation for both players (except for case.)  It would require a lot of changes to change that.  I don't mind changing things if there is a good reason but I don't like to complicate the system just to benefit one game.  I am doubtful this has any real advantage outside CwDA.  I think two letter notations would be best, but I understand that could be a radical change (needing to code in understanding of the prefix.)  I can live with heavy use of the suffixes.

I liked the idea because it creates complete freedom in the choice of id for any army, without having to worry about 'collisions' with other armies. Otherwise you would always get into trouble once the number of different armies gets large enough. It also looks a bit cumbersome to use unnecessary complex ids in a game between two given armies just because there exist other armies with which the ids would collide, while no actual collison occurs between any of the actually allowed pieces.

Oh, I certainly agree with those advantages.  I'm just concerned that this will require changes in tons of places.  Let me look and try to get a feel for just how much changing will be required...

Well, a work-around could be to define the FIDE and Clobberer Bede as different piece types, which just happen to have the same move. (And similar for Bishop.) As IDs would only occasionally clash, this means most of the time you would need 12 piece types for CwDA, instead of  10, which is probably well within the capability of ChessV. In Fairy-Max I often have to do that too, not because of the ID, but because it doesn't automatically flip the moves of black pieces. So pieces like Pawns always have to be defined as a white and a black variety. (And this posed the problem of allowing 'different' white and black pieces with the same ID.)

That is indeed a clever workaround, but it is a fairly ugly hack.  So I've started the painful process of having different piece -> notation mappings for each player.  Hopefully it won't be too bad.

I have hit my first road block however.  It seems that communication of promotion moves in CECP always uses the lower-case.  The example given in the standard documentation is "e7e8q".  Clearly, this is white's move, but the q is lower-case.  This far, I have always been sending in lower case.  How do we address this?  If I go forward with this change blindly, it will send promotion q for black but Q for white - which totally makes sense, but I'm afraid it will break engines that aren't expecting it.  An alternative, which seems ok (but please check my logic here) is to send the notation for the player which is moving, but forced to lower-case.  This shouldn't break anything, and it seems it will still work with what you are proposing.

This is indeed the way XBoard does it; for protocol moves it forces the promotion suffix to lower case, for SAN to upper case. (Also in drop moves, btw.) In these contexts case sensitivity would not provide essential info, as it is known who is on move. I don't know any variant where one can promote to pieces of the opponent's color.

For FEN XBoard strictly adheres to the convention that upper case is white and lower case is black, even when asymmetry of the armies would not make this strictly necessary because the type already implies the color.

I don't see how this interferes with white and black using different ID for the same piece type, though. If they would take the ID from the same table they would certainly have to do a case conversion for one of the colors, or either in SAN or CECP moves. That they now take it from a twice larger table without first masking away the color info doesn't change that.

Possible variant on that: use Q' etc. to indicate the pieces of black's starting army, and the plain letters for white's. That way, each symbol still means the same thing no matter which side it's on.

Or use English symbols QRBN for one color and German symbols DTLS for the other...

Ok, I've completed the change and everything seems to work.  The two sides can now have different notations for the same piece, so long as player 1's is upper-case and player 2's is lower-case.  When defining piece types when we aren't using this feature, the type notation is specified as it is for white and will automatically be changed to lower-case for black as it has always been so no existing code should break.  To specify notations when the players use different ones, just put a slash in between.  For example: "B'/b"

Here is a summary of what notations are now supported.  A notation can be a single letter.  It can be a single letter followed by ' or !.  It can also be two letters.  If the first letter of a two-letter notation conflicts with a single-letter notation, it must be prefixed by an underscore.  Use of an underscore is permitted even if there is no conflict.  The underscore is never displayed to the user.

Possible variant on that: use Q' etc. to indicate the pieces of black's starting army, and the plain letters for white's. That way, each symbol still means the same thing no matter which side it's on.

Thanks for the suggestions, but I don't think this is an improvement.  With H.G.'s approach, the apostrophe is only necessary in the rare case where a pawn promotes to a piece from the other player's army.  Also, his suggestion is backwards-compatible with Chess.  In CwDA, if both players are playing the FIDEs, everything should be identical to chess.  With this proposal, black's pieces would all have apostrophes.  Sure, you could further stipulate that the apostrophe is only used for pieces not found in the other army, but that requires additional logic for no obvious benefit.

Or use English symbols QRBN for one color and German symbols DTLS for the other

I don't think this solves any problem.  You could still have conflicts with other armies.  Besides, what's German for Bede?

Great! My kitchen is finally installed, nearly a week behind schedule due to several setbacks. (It was supposed to be finished while I was on holiday in Cyprus.) But my home is still a shambles, so it might take another week before things are back to normal here.

About the use of 'dressed letter' IDs in SAN (not really a protocol matter!): perhaps it is best to treat the punctuation just as an alternative form of disambiguation in the SAN parser. So that 'B' in the SAN move would match B, B', B", B!, ... A 'tie breaker' rule could solve cases where both a bare B and a punctuated B would match the input move, in favor of the bare B.

This would allow the user (or GUI designer) to generate the kind of SAN he likes best: either always writing puctuation, or never writing it, and rely on the usual coordinate disambiguation to indicate which piece was moved.

Congratulations on the new kitchen, and the Cyprus vacation.  That's certainly one thing I envy about Europeans - being able to pop off to so many awesome places without having to cross the Atlantic (which is a major PITA.)

Regarding SAN, for promotion, wouldn't you need to rely on presence or absense of the quote to determine which piece was being promoted to?  Note that I have no strong feelings about SAN.  ChessV doesn't do anything with it at present.

I was thinking.  If you're making a strong engine for cwda, I think I will join in and add cwda support to Quadrox.

Well, the U.S. is full of awesome places in itself. These national parks and forests are amazing. But indeed, trans-Atlantic travel is a pain.

And yes, the promotion suffix in this case would always need to be properly punctuated. This is one of the things I still have to fix in XBoard; as so far only the large Shogi variants taxed the alphabet so much that the punctuation had to be invoked, and promotion there is indicated by a '+'', I could so far ignore that problem. Except for the promotions to Buddhist Spirit / Teaching King because of 'contageon' in Maka Dai Dai Shogi. But I worked around that by using XBoards feature that allows defining a second 'nickname' ID for any piece (originally added to handle the frequently encountered non-compliant FENs for Xiangqi  that use N for H and B for E) to assign letters that otherwise were in very low demand as nickname to BS and TK, and have XBoard use the ninckname as promotion suffix when the real ID has punctuation.

Fairy-Max not supporting under-promotion, and only allowing promotion to the army's own superpiece, this problem so far did not occur in CwDA. I still have the feeling that this promotion to opponent piece types is an ugly solution to a non-existent problem, as the armies that supposedly would be disadvantaged by promotion to own pieces are already overly strong without that.

Except for the promotions to Buddhist Spirit / Teaching King because of 'contageon' in Maka Dai Dai Shogi

XBoard supports Maka Dai Dai with its 19x19 board and 50 piece types?  Holy crap!

Fairy-Max not supporting under-promotion, and only allowing promotion to the army's own superpiece, this problem so far did not occur in CwDA. I still have the feeling that this promotion to opponent piece types is an ugly solution to a non-existent problem, as the armies that supposedly would be disadvantaged by promotion to own pieces are already overly strong without that.

Yeah, I'm not sure this rule really needed to be this way.  The Nutters are the ones most likely to promote to another piece since their pieces can't move backwards quickly and so a promoted piece is stuck, greatly reducing the value of promotion.  But that army also seems to be over-powered and it is not obvious how to weaken it.  Preventing promotion to a piece from the other army might be a good way to go, but that would be a change to the game and not just a change to the army.

Any guess when you think you'll have your new cwda engine ready for testing?

XBoard supports Maka Dai Dai with its 19x19 board and 50 piece types?  Holy crap!

Well, it depends on what is meant by 'support'. Problem is that 50 is just the number of unpromoted piece types, and that there also are a lot of promoted types (although many promote to Gold, and some not at all). The standard edition of XBoard comes with SVG images for 66 piece types, and falls just short of the mark. In the development version I have increased the number of piece types, but not the number of supplied SVG images. The additional piece types thus can only be used when external piece images are supplied by the user. (WinBoard already used that strategy for a long time, as it has fewer than 66 built-in bitmaps.) The idea was that only Shogi variants have such a large number of pieces, and that people that want to play those would want to use traditional kanji pieces rather than XBoard's default pictograms, and thus would have to supply external piece images anyway. And I added special support for kanji pieces, through a general -inscriptions option, which can define an arbitrary UTF-8 string, the characters (or pairs of characters) of which will be printed on top of the corresponding pieces (upside-down for black). So all that has to be supplied is 5 or 6 SVG images of blank Shogi tiles in various sizes, and with the aid of those XBoard can be configured for any Shogi variant.

fig 1 - XBoard using kanji-inscribed Shogi-tile images.

The piece IDs and the way the pieces promote would come from the -pieceToCharTable option, or the 'setup' command received from the engine, and likewise the initial position (and board size) from the -fen option, a PGN tag, or the 'setup' command. And the way the pieces move would come from the -men option, a PGN "VariantMen" tag or the 'piece' commands received from the engine.

I must admit I did not really try that for Maka Dai Dai Shogi; just for my own shrunken (13x13) version of it, Macadamia Shogi. Which can be done with the default pictogram pieces, already in XBoard 4.9.

For WinBoard there is the 'Alien Edition'; this really does have Maka Dai Dai Shogi as a standard variant, and contains built-in kanji and move tables for all variants up to Tai Shogi, and in two of the 18 board sizes generates mnemonic piece images from the move table. This doesn't support the dressed-letter IDs, though, so FENs do not work there.

fig 2 - part of the board of the WinBoard Alien Edition set for 'variant maka' (board size 'Petite').

fig 3 - The WinBoard Alien Edition configured for Macadamia Shogi (board size 'Mediocre').

Good news.  I think I have ChessV working as a stand-alone CECP engine now.  I've played a few variants with it under Winboard and it seems to be working pretty well.  There's a little more that could be done but all the core functionality is there.

I haven't messed with programming Quadrox for CwDA yet but ChessV is ready when Kingslayer is (although I have not programmed understanding of all the endgames you've categorized yet...)  I've also been putting a lot of work into increasing its strength.  I have been playing 1600 game matches against FairyMax, Capablanca and variants, and it is now testing 106 ELO stronger than FairyMax.  I haven't tested recently, but it is probably still a good 150 ELO weaker than Joker80.

That's indeed great news! I haven't been able to do any further work on KingSlayer yet; this entire weekend I spent at the semi-annual tournament of the Dutch Computer Chess Association where my engine Spartacus scored 0 out of 7...). Now that most of the end-games have been solved, I am ready to progress with that. 'Army picking' is an issue that I have no clear idea on how exactly to implement, though. Of course I want the engine to be able to do games between all the standard armies, controlled by the CECP name of the (sub-)variant (like cwda~nutters~fide).

But I want it to be able to do more than that, because my main reason for converting KingSlayer was to be able to improve the accuracy of piece-value measurements over what Fairy-Max was capable of, by taking into account pair bonuses and mating potential in the eval, and providing a generally better level of play (to be exploited by being able to do more games of the same quality in the same time). For this reason I want to embed pieces in armies to which they do not belong, so that they are part of the only imbalance in otherwise symmetric setups. This requires 5 piece types per player (plus Pawns plus King), which should not be a major problem, since the internal piece encoding uses 3 bits for piece type (while white and black pieces are independently specified), code 0 being reserved for empty squares.

So I would need some manner to specify 5 pieces for each side, taken from the total collection of all supported pieces, independently. This would not be so much of a problem for properties of a single piece, (such as value, mating potential or color binding) which could just be copied from a table. But it is more problematic to get properties concerning multiple pieces, such as the winning prospects of 5-men end-games. If armies can be arbitrarily composed there would be 16000 of those (for 5 armies), most of which I have not calculated yet because they would consist of unnatural or impossible combination. And I want to be able to test pieces not in any of these armies as well (such as R3, RF).

Perhaps the best method would be to program a heuristic for determining the winning-prospects based on the individual properties of the participating pieces (piece value, mating potential, color binding) that roughly captures the general trends that were revealed by the EGT generation I did so far, and then supplement that with a list of exceptions located in an external file. To be fully accurate it would be enough to only have the exceptions in that file that can occur with the piece sets currently in use (which, for unconventional combinations, could be calculated by EGT generation for the purpose, if not yet available).

Perhaps I should define a general variant 'cwda', which would be entirely configurable through an external file, the latter containing the info which of the programmed pieces occur in either army, and under which single-letter ID. Where piece number 1-20 are those of the 5 standard armies, (hard-coded in KingSlayer) and 21-26 would be configurable through the same file (as any combination of possibly divergent finite-range slides).

You are taking on a couple of difficult problems ...

Being able to mix-and-match pieces is obviously difficult and the CECP isn't really suitable for this.  External configuration with a text file is probably what is required.  It wouldn't need to be all that sophisticated if your configurablity is limited to adding new piece types.  ChessV does offer a simple scripting language that will let you accomplish this.  What it supports is hit-and-miss but just adding new (basic) piece types is no problem.  Your ski-jump move is not presently supported however.  I need to add that anyway for the Grand Accetrix Griffin, but it won't make this next release.  It has been almost a year and a half since I've put out a new version and I have added so much new stuff since then that I just need to wrap up and get it out.

Knowing when to end a game due to insufficient mating material and when to return a zero evaluation is also obviously tricky.  The currently released ChessV 2.1 doesn't do anything about this at all.  IIRC, you can be down to two kings and it'll happily wander around until the 50-move rule is triggered.  The new version is somewhat better.  It knows about endings with standard chess pieces, and even includes customized evaluation for a few combinations (KRKP, KRKB, and KRKN), plus a "mop-up" evalution for lone king vs. rook or more.  But throw in any non-standard pieces and it goes back to not making any decisions... King + Camel vs. King and you are back to wandering around for 50 moves with it stupidly thinking the side with the Camel is ahead (although it is smart enough to know that if the other side has a pawn then the side without and less than a rook's worth of material should be scaled down massively.)  I would like to program in more smarts here, too, especially in light of all your work to quantify these endings.  I now have a material hashtable so it could be done without worrying about computational cost, but probably won't get to too much of this for this version.  Looking for low-hanging-fruit, like a simple piece property that says "this piece can't mate without help."  It would be really, really awesome to make it smart enough that, when a game starts, even if it is a user-define game through an include file, it does enough endgame table generation to make some basic decisions about this stuff, but that is probably a long way off.

It would be really, really awesome to make it smart enough that, when a game starts, even if it is a user-define game through an include file, it does enough endgame table generation to make some basic decisions about this stuff, but that is probably a long way off.

I think Sjaak II makes an attempt at that. Of course there are some cases where this is obvious, such as not attacking any orthogonally adjacent squares. Beyond that heuristics quickly get as complicated as a single iteration of EGT generation, meaning that code-wise there is litlle to gain, and you might as well put the loop around it to do the full generation, if you can afford it timewise. Without full generation you would probably never see that a Silver General has no mating potential, or that the forced mate with  a 'Turtle' (fhNfAbF) is almost entirely cursed because it takes on average 100 moves.

This is not out of the question; even the JavaScript generator in the checkmating applet typically needs under 100 msec to solve a 3-men 8x8 ending. And you could have the engine maintain a file where it 'caches' the conclusions from such EGT generation on disk. It could consult that file whenever a new variant is selected (or even just at the start of every game), to see if it contains data on all of the participating pieces, and if not, generate the EGT and add the data. With such a design it could even be acceptable to generate all 4-men of the participating pieces, to know what in general draws what, and which pairs of minors can checkmate. It only means that first time you use a newly defined variant it takes a few minutes to start.

A nasty aspect of EGT generation is that you don't just need the normal moves of the piece, but also the retrograde moves. For point-symmetric leapers and sliders these are the same, and even for asymmetric pieces you can just use the flipped version of the piece for the retrograde moves. But for bent sliders the moves are fundamentally different, and would need a separate provision in the move generator.

I now managed to have KingSlayer play some CwDA games against Fairy-Max, which it eventually could do without illegal-move complaints (and even win some). This is just the basics (move generator and check test); the material table is not implemented yet. The non-standard a-side castling of the Clobberers works, though. No bent or ski-sliders yet, so also no Dragons Army.

This already touched on some compatibility issues. Like Fairy-Max KingSlayer now supports two methods for variant selection: in variant 'fairy' the armies are decided based on White Army / Black Army engine options. This is compatible with Fairy-Max, which has a single engine option to select a pair of armies, but the options have to be set by hand anyway, and mean nothing to WinBoard. The other method is through the variant names, but the names are incompatible with those in use by Fairy-Max; I currently programmed names like "fide~nutters~(cwda)" (all lower case; the space existing WinBoard versions reserve for these names in the New Variant dialog is rather small, and capitals take more space). This we obviously have to sync between ChessV, Quaddrox and KingSlayer, and then I will adapt the names in Fairy-Max' ini file likewise.

The other issue is the piece IDs. I currently use the same as Fairy-Max already used. These are not very satisfactory, though; many were picked to match the piece symbol I picked to represent the piece in WinBoard (e.g. U for Charging Knight because I displayed it as a Unicorn, E for Waffle because I displayed it as an Elephant). They make no sense in a GUI that would use a less fanciful representation.

Promotion to an opponent piece will always be somewhat problematic in KingSlayer, as it can only support 7 piece types per army (including Pawn and King), now that I patched it to allow use of the formerly unused type code 7. So that could be its own pieces plus the opponent's super-piece. But then under-promotion to an opponent piece will always be impossible. This is not very important for the purpose I wanted to use it for, but it makes it a somewhat incomplete implementation of the official rules. Things could be stretched a bit further by dynamically assigning the piece type during the game. But that would not remove all limitations, and add a lot of cumbersome code for something that would be almost entirely hypothetical.

I also discovered a problem in WinBoard: it does not relay castling rights in a FEN when the corner piece is not a Rook. This never hurt Fairy-Max, as this relies on the 'edit' command rather than 'setboard', which does not relay any info on castling rights. So Fairy-Max would reconstruct these just based on the corner piece being the one from the initial setup. KingSlayer takes the rights in the FEN seriously, though, and in an engine-engine game the initial setup specified by the first engine would be loaded in the second engine, which then would lose its castling rights. (With external configuration of the initial position both engines would lose these rights, if they rely on 'setboard'.) There is a WinBoard command-line option that can force the castling field to a desired value, though (to force compatibility with the broken Arena Chess960 FEN), so I suppose that would be a work-around. Obviously I have to fix this in WinBoard.

I now managed to have KingSlayer play some CwDA games against Fairy-Max, which it eventually could do without illegal-move complaints (and even win some). This is just the basics (move generator and check test); the material table is not implemented yet. The non-standard a-side castling of the Clobberers works, though.

Nice.  That's progress :)

I currently programmed names like "fide~nutters~(cwda)" (all lower case; the space existing WinBoard versions reserve for these names in the New Variant dialog is rather small, and capitals take more space). This we obviously have to sync between ChessV, Quaddrox and KingSlayer, and then I will adapt the names in Fairy-Max' ini file likewise.

That's fine.  I will present the formal names to the user so it doesn't matter much to me what is sent to the engine in the backgrond.  I will synch to whatever you settle on.

The other issue is the piece IDs. I currently use the same as Fairy-Max already used. These are not very satisfactory, though; many were picked to match the piece symbol I picked to represent the piece in WinBoard (e.g. U for Charging Knight because I displayed it as a Unicorn, E for Waffle because I displayed it as an Elephant). They make no sense in a GUI that would use a less fanciful representation.

Yeah, there is that too.  But I'm not sure what the "correct" notations would be anyway.  I have assigned letters already but you probably won't like them.  I abandoned Betza's terrible, horrible, absolutely no good names at least a decade ago, using alternatives when they already existed.  The Waffle?  For real?  People will not take seriously a game with a piece called the Waffle...  And that piece is already know by the far more satisfactory "Phoenix".  Betza had already started fixing the names - the "Furhrurlbakking" becoming the Colonel for example.  Too bad he disappeared before he finished.

I patched it to allow use of the formerly unused type code 7. So that could be its own pieces plus the opponent's super-piece. But then under-promotion to an opponent piece will always be impossible.

This is not ideal, but is probably not all that big a deal.  How often will any engine pick something other than the two super-pieces?  I would think pretty rare, and then 50% of the time it will still pick one of its own lesser pieces.

But I'm not sure what the "correct" notations would be anyway.  I have assigned letters already but you probably won't like them.  I abandoned Betza's terrible, horrible, absolutely no good names at least a decade ago, using alternatives when they already existed. 

I completely agree. I am all in for renaming them, but I do think it is important we do it in the same way. 'Phoenix' is great, but what ID do you use, as the P is already taken? I would use 'X', as that is otherwise in little demand. 'Short Rook' (S) for the R4 would also be an obvious choice. I always liked the name 'Tower' for Rookish pieces (as there is also little pressure on T), but am not sure whether that would best fit the WD or the Charging Rook. 'Lighthouse' would also be a suitable name for an augmented Rook. 'Duck' for the HFD seems as good a name as any. Fairy-Max used 'Unicorn' for the Charging Knight, but this has already a Grant Acedrex meaning. How about 'Jockey' for a Horse/Man-like piece (other than the KN Centaur)? The Bede should probably have the name of some sort of (rather important) cleric. How about 'Ayatollah'? The FAD gives the impression of nervously jumping around, but the name Squirrel is already taken. It can be seen as an augmented Modern Elephant, but that seems a misnomer for a jumping piece in the first place. 'Kangaroo' would be fitting, but has occasionally been used for several other pieces.

Excellent.  I'm glad you're on board.  And I did, indeed, use X for the Phoenix.  I think I used S for Short Rook also if memory serves.  I have to head to work so I don't have time go into too much detail about what I chose and why right now.  My goal was to not invent names myself any more than absolutely necessary - I used existing naming wherever I could find one.

Funny you mention "an important cleric" for the BD.  The name I have been using is "Cleric".  This is a case where I was not aware of any existing name and Cleric was a nice medieval name that hadn't been used.  For FAD I used "War Elephant" - the piece is an elephant crossed with a dabbabah (war machine.)  I didn't invent that, but I don't remember exactly from where it came.  I think Joe Joyce may have come up with that originally.  The Fibnif, I used "Narrow Knight".  This just seemed obvious and similar to Charging Rook and Charging Knight, who's names I didn't feel needed changing.  The HFD is the "Lion".  This was originaly used by David Paulowich in his games and I followed his lead in my games (Opulent Chess and Hubbub.) This is one of my favorite variant pieces.  Yes, the Lion has a different meaning in Chu Shogi, but that piece is pretty different and wasn't really used in conventional chess variants.  You have since used it in Elven Chess with a different name, but that was long after.  David used this name about 15 years ago.  Then there's the BD.  I have been using "Bowman" but I was never all that happy with this one.  It came from Compton Medieval Chess, which was the only existing use of the piece I could find, but this game is extrememly obscure and the name didn't really seem to match the piece.  Tower is probably a better name and I'm happy to change.

How about this? I changed Bowman to Tower, which is probably a little better. It is unfortunate that I cannot find a historical precident. It seems like this is an obvious piece that would have been used log ago but apparently not. For the Nutty Knights, We have Charging Rook, Charging Knight, and Colonel all beginning with C. The Charging Rook can be R. The Charging Knight could be N, but then what about Narrow Knight? I think Jockey sounds like a reasonable choice for the Charging Knight (a piece that moves partly like a horse and partly like a man) and J is unused. Leaving us with:

I'm also open to something other than Narrow Knight if we can come up with a GOOD name that makes sense. Although, if we did that, that would free up N for Charging Knight, which would have the advantage of not reassigning names to those pieces that Betza already fixed. I suppose that the Fibnif could be Jockey since it also has horse-moves and man-moves, although not as obviously as the Charging Knight.

A good name for the narrow knight could be searched in the realm of light cavalry, maybe. Lancer or hussar could be interesting options. Donkey is a fun option also for the not this "not that horse" piece, but maybe this is not that appropriate. 

"Lancer" does seem a good name, as a Lance seems to be a weapon ill-suited for lateral combat.

I like Lancer as well.  So the Fibnif becomes Lancer (L), freeing up N for the Charging Knight.  Now we are only changing the names of pieces Betza didn't get around to yet and we still have pretty good one-letter notations.

I have updated the table below.

Wouldn't it be more logical to use 'E' for the War Elephant? It is likely to be depicted as an Elephant, so this would have a better mnemonic value.

You make a strong case for 'Lion'. (And it indeed is a very nice piece.) I suppose the reason I don't like it is that I am just too involved with Shogi, and that the Lion is such an iconic piece there. But I guess I just should come to terms with the fact that Shogi names often have completely different meaning than those in western chess variants, and that a (Drunk) Elephant is nothing like an Alfil, and a (Dragon) Horse nothing like a Knight.

@HG

I had sugested, for one of his variants, the name platinum general for the drunk elephant. To me that seems more interesting. But that's a bit off topic!...

Wouldn't it be more logical to use 'E' for the War Elephant? It is likely to be depicted as an Elephant, so this would have a better mnemonic value.

Yup, I agree with this.  I have updated the table.

Just let me know what exactly you want the xboard game names to look like.  I was thinking we could extend your idea a little and say that where game-specific parameters need to be encoded in the game identifier, they are formatted like P1(game) or P1~P2(game) or P1~P2~P3(game), etc.  CwDA has two parameters, P1 is white army and P2 is black army.  At some point I would like to come up with a better option for programs that support it so we don't have an exponential explosion of game names, but as a starting point, and for backwards-compatability with programs that don't support it, this seems like a promising approach.

I'm really looking forwards to having multiple interoperable CwDA engines.  I originally started writing ChessV because I wanted to better study CwDA and Zillions wasn't good enough (and there was nothing else.)

I currently put an extra ~ in the names compared to what I proposed here earlier: A~B~(C) instead of A~B(C). This looked more balanced (the latter form suggested a tighter coupling between B and C). This would also make it possible to recognize '1-parameter' sub-variants by the ~ alone, e.g. bird~(10x8), carrera~(10x8).

Name explosion would occur when a sub-variant needs multiple parameters (such as CwDA); with a single parameter the possible parameter values would have to be mentioned anyway. We could solve it by introducing wildcards for the parameters: nutters~*~(cwda) would mean the variant group 'cwda' has two parameters, and 'nutters' would be a possible value for the first parameter. Such a partial variant declaration would be discarded if there weren't also declarations of the form *~rookies~(cwda) in the variants feature to specify some possible values for the second parameter. This is not backward compatible with existing GUIs, though; these would not recognize the * as wildcard, and take the names litterally. This problem could be ameliorated by making the engine such that it would also accept the names with the wildcard in it in the 'variant' command, and would treat the parameter value * as a command to keep the previous value of that parameter. A user could then still make the selection Nutters vs Rookies by first selecting nutters~*~(cwda), and then switch to *~rookies~(cwda). (On second thought, we could adopt the convention that a second parameter that is only given as a wildcard in all the supported sub-variants would be assumed by the GUI to allow the same values as the first parameter. The engine, when confronted with * for this parameter, could then use the previous value of the first parameter for it.)

BTW, many of the problems you mention for Cylinder Chess sound familiar. I remember that at some point of implementing XBetza support in XBoard I decided moves should be generated after 'lifting' the moving piece off the board, in order to allow bent multi-leg moves to pass back over itself (e.g. for igui). That caused XBoard to hang with an oR on an empty rank... I solved it by making the Betza-driven move generator such that it would treat all slides as finite-range slides with the maximum board dimension minus 1 as range. If you already support finite range slides, this would cause no extra overhead.

SEE is indeed troublesome, if there are pieces with fancy moves. But like you say, it isn't really necessary. Pure MVV/LVA already is pretty good, and Fairy-Max gets by with only sorting the MVV/LVA-wise best move in front, and searching the other captures in arbitrary order. My guess is that >90% of the beneficial effect of SEE comes from detecting if a piece is completely unprotected, so that HxL captures on it are safe. So I often use the simpler algorithm BLIND, which postpones HxL capture of protected pieces compared to MVV/LVA. This still requires you to know whether a piece is protected, though. The null-move reply could have told you, but in QS there is no null move. You could get (somewhat inaccurate) protection information as a side effect from the move generation that was done in the parent; this could have marked all pieces that block friendly captures as protected (e.g. in a bitmap of the board or of the piece list, or even a (packed?) set of counters, to be able to correct single protection for the piece that was moved).

KingSlayer uses MVV/LVA, but it has a provision to initially abort the search of HxL captures that go sour, to reschedule them until after the other captures have been searched. To this end it can pass the value of the victim as 'threshold' to the child. When that child then generates a capture of a piece more valuable than 'threshold' on that same square (which it would after H x protected L), it immediately returns 'abort score' INF+1 (so the caller can see that the capture was bad and needs to be rescheduled). As an extra it would also abort if the 'obvious gain' from capture on a different square would exceed 'threshold'. Where that gain is defined as the value of the victim if unprotected, or the difference between victim and attacker value if protected. This protection information is obtained from the parent (inaccurate as it is, but only used in 'second order'). Normally you would only abort on capture of a King (returning +INF, as no rescheduling is needed), i.e. use threshold = INF-1. This is what is done during the search of the rescheduled captures, to prevent it would abort again.

Duplicat moves are not really a very large problem; the duplicat should give a hash hit that makes it fail low immediately. Bifurcating pieces (e.g. bent sliders like the Griffon) also tend to produce them, if you are not careful to define the overlapping part of one of the branches as a lame leap.

Bent sliders (including the ski-jump case) are pretty nasty anyway. They blur the distinction single vs double check, as beside offering the possibility for triple (or even quadruple) check, they can cause double check to occur along a single path. So that contrary to normal double check, it can be resolved by interposition (but still not by capture of the checker). The ski-slide of the Wyvern thoroughly wrecks KingSlayer's implementation of check detection and handling. Perhaps I should not spent further time on that now, and try to get the other armies working as quickly as possible.

In that case I am nearly there; I have implemented a rather general material evaluation that reproduces the general trend of the EGT results, driven by a 32-bit 'properties' word for each piece type. The even bits in this word are used as flags to indicate whether that piece has the corresponding property (e.g. mating potential, color binding, lacking mating potential even as a pair, being rather weak for a major, etc.). As I won't do any score discounting with more than 2 pieces for each side, the property words can be added for each player, and when both pieces have a certain property, this cause a carry to the (originally unused) next-higher (odd) bit. This makes it easy to recognize conditions like having two majors (almost always a win), having a pair of Knights (draw), having only minors (draw against any defender), etc. Pieces with the 'super-piece' property will use a 2-bit field in the properties word to indicate their ranking, so that we can decide whether an extra minor on their side will make it a win.

It also detects uneven distribution of color-bound pieces over the square shades, to penalize that by a fixed score (which is equivalent to awarding a pair bonus), and declaring a draw when it happens to the only two pieces in absence of Pawns. And when both sides have full color binding on opposite shades it will discount the evaluation by a factor 2 even with many Pawns. This seems to capture the most important effects. I guess I could still have it invoke the existing code for detecting a KBP.K draw in all cases where the only remaining piece is color bound.

Still have to test everything, though.

OK, I seem to have produced something that works according to specs. I uploaded it to http://hgm.nubati.net/CwDA.exe. I had it play matches against Fairy-Max in FIDE-Nutters and Clobberers-Rookies, and there don't seem to be any illegal moves anymore. I did this using the old piece IDs of Fairy-Max, but in the uploaded version I replaced this by the new IDs we agreed on, but added a checkbox option that can be used to select the old IDs. The piece-to-char tables sent in the 'setup' command still always use the old IDs, though, and don't provide IDs for pieces of the opponent army; this is still something I have to fix. But I am not sure ChessV even looks at the setup command, and the pieceToCharTables probably have meaning for Win/XBoard only.

I also added a checkbox option 'Traitor promotion', where you can switch the possibility to have it promote to the super-piece of the opponent army on or off.

[Edit 2019-05-28] I fixed a problem with input of under-promotions, (which could not be tested by playing against Fairy-Max, as it never does those). It now also can do all under-promotions to pieces of its own army (the FIDE version of KingSlayer only considered promotion to Q and N, but not in all armies the other pieces are as redundant as B or R). The pieceToCharTables in the 'setup' command now use the piece IDs that are selected by the old IDs option, and merge in the super-piece of the opponent army, if the 'Traitor promotion' option is enabled.

The previous version was still plagued by a woken-up 'sleeper bug': all the end-game discounting worked fine when I tested it on setup positions for the end-game, but it did not apply it when the same end-game was reached in games! Turned out castling messed up the Rook counter, which had never mattered before, as the count was not used. But now it made the drawishness code think both sides had (tons of) Rooks. I uploaded a fixed version ("KingSlayer cwda-1.2") to the same link. This also prints some results of the drawishness detection for the root as part of the Thinking Output, so that it can be easily seen when it misjudges. Like Fairy-Max it slightly randomizes the first 8 ply of any game, (adding -8 to +7 cP to the score of each move in the root), to provide more game diversity in test runs.

Nice!  I didn't have an opportunity to mess with this during the week, but I now have KingSlayer cwda integrated within ChessV cleanly.  It is pretty strong!

I am finding one issue, though.  Sometimes upon promotion KingSlayer sends the wrong notation. For example, ChessV was playing the Clobberers with the white pieces and KS was playing the FIDEs with the black pieces.  When it went to promote, it sent c2c1q' which is not correct.  The Queen is part of its army, not the opponents, so the quote shouldn't be there.  (ChessV then says it forfeited due to illegal move.)

A yes, this was something I couldn't test, because WinBoard doesn't accept dressed letters as promotion suffix yet. This was a bit of a subtle bug: the quote suffix was initialized to 0 when entering the MoveToText routine, and then set to a quote if promotion to the 7th piece type occurred before being printed. But by an oversight the suffix had been declared as a static char, so the initialization did not work on every call of MoveToText, but only at program start. So once a move with quote suffix was printed, all promotions would from then on be printed with this suffix. (But WinBoard happily ignored that.)

I uploaded a fixed version to the same link.

Sorry, H.G., it looks like that did not solve the problem.  I am also getting an occasional crash.  I'll email you the debug output.  But when it doesn't crash or make illegal moves, it seems to win the majority of the time :)

Also, it looks like it doesn't support both sides playing the same army?

Are you sure you got the new version, and that there was no problem with your browser caching the old version? It should say cwda-1.3 for the version number, and have 'dragons' as one of the options for the White/Black Army (not properly working yet, though). I tested it with a position "8/6P1/6k1/8/8/8/8/1K6 w - - 0 1" in FIDE vs Rookies, and it does play g7g8q without quote in that position.

In earlier versions I had some crashes too, which seemed to occur when the score got extreme. This disappeared after I calculated the interpolation between opening and end-game evaluation in a less overflow-prone way, and the version I originally posted had played several hundred games against Fairy-Max without a single crash. It could be that I broke something when changing the check test to also work with ski-slides. (But I see in the log that this happened for 1.2, so this cannot be the explanation.) This crash also doesn't reproduce.

Indeed I did not put equal armies in the list it announces in the variants feature. It would recognize the names in the 'variant' command, but of course a proper GUI would never send those if the engine did not announce it supported those. It is possible to play equal armies by selecting those through the White/Black Army options, and then select variant fairy. (This is how I test, as it currently is the only method where it is compatible with Fairy-Max, when I tick the option for old piece names, and turn traitor promotion off.)

BTW, in the log I see that you did not send a 'memory' command to set the hash size. That means KingSlayer would use its default hash size, which is only 1MB.

Yes, you're right, I was still on 1.2 somehow.  I will test with 1.3.

I have had some more time to test.  This is a really an excellent CwDA engine.  I think I have found another bug though.  In one game, it insisted ChessV made an illegal move when the move was fine.  In that game, ChessV was white playing the nutters and KS was black playing the rookies.  KS promoted a pawn on g1 and chose to promote to a colonel.  ChessV then moved the king to g4 and KS called this an illegal move, presumably thinking that its colonel on g1 attacked it.

So I think the issue is that when KS plays black and promotes a pawn to a piece from white's army, it thinks that the piece moves "forward" as a white piece instead of as a black piece.

Also, good catch about the memory usage.  ChessV does not yet offer an xboard engine's configuration options but as a stop-gap I have added settings for the generic "memory" and "cores" options and it will supply them to xboard engines that support those featuers.

Your diagnosis is entirely correct. Because KingSlayer was originally designed with only (2x) 7 piece types, it initializes the move-generator data for the four variable types from a larger list of all types depending on the army choice. But when 'Traitor promotion' is set it initializes the 5th variable type as a copy of the 4th, in the table for the opponent. But I forgot to flip the orientation. I will add a note to this posting when I have fixed it.

Good that ChessV now gives a memory command. I already had made KingSlayer a bit more robust to not receiving one, by having it interpret memory size 0 as 128KB and launching it with that size, and then check the memory size at the start of every search to see if it is still at 0, and allocating 64MB if it is. But relying on default hash sizes is likely to give rise to unfair use of resources.

[Edit] OK, I uploaded a version (0.1, same link) where the traitor orientation should be fixed. This was a bit tricky, as the initialization also collects all sliding directions of an army, to know from which directions it should expect discovered checks.