Siderite's Blog

and has 0 comments

Here is the scenario:

  • you have two tables: a source of recent data and an SQL Server destination table with the same structure that has to be updated with that data
  • they are both on the same server (we assume that if the data source is external you have copied it into a local temporary table)
  • the destination table is in active use
  • the source destination may be in active use also
  • the tables are large, locking them and waiting for the merge to finish is not an option
  • the difference between the tables is usually small. Not so small as to be irrelevant, but small relative to the size of the tables
  • we don't care about full data consistency (any intermediate state between the current and the updated one is acceptable)

There are some quirks in SQL Server that affect our task, mainly that if you try to modify 5000 rows or more of a table, then the locking will escalate from row locks to a table lock, turning it unusable during the merge. Also, for operations involving a lot of data, the log of the database will increase to accommodate that size, then the changes will be persisted and the log invalidated. This takes a lot of disk space and if anything happens during the operation, the entire operation will be rolled back, with all of the data in the log having to be restored, which also takes a lot of time, blocking the database.

The first idea is to find rows that are different, new or missing and just sync 5000 of them at a time. One can do this with a MERGE or classic INSERT, UPDATE, DELETE operations. And this works, but it has a major flaw: after the first 5000 rows have been found and synchronized, the next operation will go through them again anyway to find the next 5000 rows, and again, and again, and again. The execution time will increase exponentially with every new batch of rows.

What we actually need is to take a list of changes and apply them in batches, kind of like an artificial transaction log. The following stored procedure will take two parameters: the full schema name of the source table and the full schema name of the destination table. It will create a log of changes, take 4900 of them at a time and apply them to the destination table. The only restrictions are that the destination table has to have primary key columns and the source and destination tables have the same columns. For performance reasons, it's best that the source table also has the same primary keys or at least an index on the same columns. The usage would look like EXEC usp_MergeTables 'SomeSchemaIncludingDbo.SourceTable','MaybeSomeOtherSchema.DestinationTable'

I will explain what it does after the code:

-- this is for batched merge (del, upd, ins) of the data of a table into another (nolock/readpast for reduced locking of source and destination tables)

CREATE OR ALTER PROC usp_MergeTables(@SourceTable NVARCHAR(256),@DestinationTable NVARCHAR(256))
AS
BEGIN

SET NOCOUNT ON

IF NOT EXISTS(SELECT * 
FROM sys.schemas s
INNER JOIN sys.tables t
ON s.schema_id=t.schema_id
WHERE REPLACE(REPLACE(@SourceTable,']',''),'[','')=s.name+'.'+t.name)
BEGIN

  DECLARE @Err1 VARCHAR(100) = 'Source table '+@SourceTable+' not found!'
  ;THROW 50404,@Err1,1

END

SELECT CAST(s.name as NVARCHAR(Max)) as schemaName,CAST(t.name as NVARCHAR(Max)) as tableName,CAST(c.name as NVARCHAR(Max)) as columnName,c.is_computed,c.is_identity, 
CASE 
  WHEN tp.name IN ( 'varchar', 'char', 'varbinary' ) THEN tp.name + 
                  CASE WHEN c.max_length = -1 THEN '(max)'       
                         ELSE '(' + CAST(c.max_length AS VARCHAR(4)) + ')' END       
  --types that have an unicode character type that requires length to be halved 
  WHEN tp.name IN ( 'nvarchar', 'nchar' ) THEN tp.name + 
                   CASE WHEN c.max_length = -1 THEN '(max)'       
                         ELSE '(' + CAST(c.max_length / 2 AS VARCHAR(4)) + ')'       
                                                              END
   --types with a datetime precision 
  WHEN tp.name IN ( 'time', 'datetime2', 'datetimeoffset' ) THEN tp.name + 
                                            '(' + CAST(c.scale AS VARCHAR(4)) + ')' 
  --types with a precision/scale 
  WHEN tp.name IN ( 'numeric', 'decimal' ) 
  THEN tp.name + '(' + CAST(c.precision AS VARCHAR(4)) + ',' +
                                             CAST(c.scale AS VARCHAR(4)) + ')'
  --timestamp should be reported as rowversion 
  WHEN tp.name = 'timestamp' THEN 'rowversion' 
  --and the rest. Note, float is declared with a bit length, but is 
  --represented as either float or real in types  
  ELSE tp.name 
END as typeName,
ISNULL(( SELECT pk.is_primary_key FROM sys.indexes pk
INNER JOIN sys.index_columns ic
ON ic.object_id = pk.object_id
AND ic.index_id = pk.index_id
AND c.column_id=ic.column_id
WHERE t.object_id = pk.object_id
AND pk.is_primary_key = 1
),0) as is_primary_key
INTO #tgtColumns
FROM sys.schemas s
INNER JOIN sys.tables t
ON s.schema_id=t.schema_id
INNER JOIN sys.all_columns c
ON t.object_id=c.object_id
INNER JOIN sys.types tp
ON c.system_type_id=tp.system_type_id
AND c.user_type_id=tp.user_type_id
WHERE REPLACE(REPLACE(@DestinationTable,']',''),'[','')=s.name+'.'+t.name

IF NOT EXISTS(SELECT * FROM #tgtColumns)
BEGIN

  DECLARE @Err2 VARCHAR(100) = 'Destination table '+@DestinationTable+' not found!'
  ;THROW 50404,@Err2,2

END

IF NOT EXISTS(SELECT * FROM #tgtColumns WHERE is_primary_key=1)
BEGIN

  DECLARE @Err3 VARCHAR(100) = 'Destination table '+@DestinationTable+' has no primary keys!'
  ;THROW 50404,@Err3,3

END

DECLARE @operSql NVARCHAR(Max)
DECLARE @delSql NVARCHAR(Max)
DECLARE @updSql NVARCHAR(Max)
DECLARE @insSql NVARCHAR(Max)
DECLARE @identityInsertOn NVARCHAR(Max)=''
DECLARE @identityInsertOff NVARCHAR(Max)=''

IF EXISTS(SELECT * FROM #tgtColumns WHERE is_identity=1)
BEGIN

  SET @identityInsertOn=CONCAT(N'
SET IDENTITY_INSERT ',@DestinationTable,N' ON
')
  SET @identityInsertOff=CONCAT(N'
SET IDENTITY_INSERT ',@DestinationTable,N' OFF
')

END


SELECT @operSql = CONCAT(N'DROP TABLE IF EXISTS #oper

SELECT *
   INTO #oper
FROM (
SELECT ',STRING_AGG(CONCAT(N'ISNULL(src.[',c.columnName,N'],tgt.[',c.columnName,N']) as [',c.columnName,N']'),N', '),N',
  CASE
    WHEN ',STRING_AGG(CONCAT(N'src.[',c.columnName,N'] IS NULL'),N' AND '),N' THEN ''DEL''
    WHEN ',STRING_AGG(CONCAT(N'tgt.[',c.columnName,N'] IS NULL'),N' AND '),N' THEN ''INS''
    WHEN (
    	SELECT * FROM ',@SourceTable,N' R
    	WHERE ',STRING_AGG(CONCAT('R.[',c.columnName,N'] = src.[',c.columnName,N']'),N' AND '),N'
    	FOR XML PATH(''Row''), ELEMENTS XSINIL
      ) <> (
    	SELECT * FROM ',@DestinationTable,N' R
    	WHERE ',STRING_AGG(CONCAT('R.[',c.columnName,N'] = tgt.[',c.columnName,N']'),N' AND '),N'
    	FOR XML PATH(''Row''), ELEMENTS XSINIL
      ) THEN ''UPD''
   END as __oper__
   FROM ',@SourceTable,N' src (NOLOCK)
   FULL OUTER JOIN ',@DestinationTable,N' tgt (NOLOCK)
   ON ',STRING_AGG(CONCAT('src.[',c.columnName,N'] = tgt.[',c.columnName,N']'),N' AND '),'
) x
WHERE __oper__ IS NOT NULL

CREATE INDEX temp_id ON #oper(',STRING_AGG(CONCAT('[',c.columnName,N']'),N', '),N')
CREATE INDEX temp_oper ON #oper(__oper__)')
FROM #tgtColumns c
WHERE c.is_primary_key=1

SELECT @delSql = CONCAT(N'

DECLARE @batch TABLE(',STRING_AGG(CONCAT('[',c.columnName,N'] ',c.typeName),N', '),N',
                   PRIMARY KEY(',STRING_AGG(CONCAT('[',c.columnName,N']'),N', '),N'))

DECLARE @ROWS INT = 1

WHILE (@ROWS>0)
BEGIN

  DELETE TOP (4900) tgt
    OUTPUT ',STRING_AGG(CONCAT('deleted.[',c.columnName,N']'),N', '),N'
	INTO @batch
  FROM ',@DestinationTable,N' tgt (READPAST)
  INNER JOIN #oper src
    ON ',STRING_AGG(CONCAT('src.[',c.columnName,N'] = tgt.[',c.columnName,N']'),N' AND '),N'
  WHERE src.__oper__=''DEL''

  SET @ROWS=@@ROWCOUNT

  DELETE o
  FROM #oper o
  INNER JOIN  @batch b
  ON ',STRING_AGG(CONCAT('o.[',c.columnName,N'] = b.[',c.columnName,N']'),N' AND '),N'
  
  DELETE FROM @batch

  IF (@ROWS=0)
    SELECT @ROWS=COUNT(*) FROM #oper WHERE __oper__=''DEL''

END')
FROM #tgtColumns c
WHERE c.is_primary_key=1


SELECT @updSql = CONCAT(N'

SET @ROWS = 1

WHILE (@ROWS>0)
BEGIN

  UPDATE tgt
  SET ',(SELECT STRING_AGG(CONCAT('[',c.columnName,N'] = src.[',c.columnName,N']'),N', ')
FROM #tgtColumns c
WHERE c.is_primary_key=0 AND c.is_computed=0),N' OUTPUT ',STRING_AGG(CONCAT('inserted.[',c.columnName,N']'),N', '),N'
	INTO @batch
  FROM ',@DestinationTable,N' tgt (READPAST)
  INNER JOIN ( 
    SELECT TOP (4900) s.*
    FROM #oper o
	INNER JOIN ',@SourceTable,N' s
	ON ',STRING_AGG(CONCAT('s.[',c.columnName,N'] = o.[',c.columnName,N']'),N' AND '),N'
    WHERE __oper__=''UPD''
  ) src
    ON ',STRING_AGG(CONCAT('src.[',c.columnName,N'] = tgt.[',c.columnName,N']'),N' AND '),N'

  SET @ROWS=@@ROWCOUNT

  DELETE o
  FROM #oper o
  INNER JOIN  @batch b
  ON ',STRING_AGG(CONCAT('o.[',c.columnName,N'] = b.[',c.columnName,N']'),N' AND '),N'
  
  DELETE FROM @batch

  IF (@ROWS=0)
    SELECT @ROWS=COUNT(*) FROM #oper WHERE __oper__=''UPD''

END')
FROM #tgtColumns c
WHERE c.is_primary_key=1


SELECT @insSql = CONCAT(N'

SET @ROWS = 1

WHILE (@ROWS>0)
BEGIN

  INSERT INTO ',@DestinationTable,N'(',(SELECT STRING_AGG(CONCAT('[',c.columnName,N']'),N', ')
FROM #tgtColumns c
WHERE c.is_computed=0),N') OUTPUT ',STRING_AGG(CONCAT('inserted.[',c.columnName,N']'),N', '),N'
    INTO @batch
  SELECT TOP (4900) ',(SELECT STRING_AGG(CONCAT('s.[',c.columnName,N']'),N', ')
FROM #tgtColumns c
WHERE c.is_computed=0),N'
  FROM #oper o
  INNER JOIN ',@SourceTable,N' s
  ON ',STRING_AGG(CONCAT('s.[',c.columnName,N'] = o.[',c.columnName,N']'),N' AND '),N'
  WHERE __oper__=''INS''

    SET @ROWS=@@ROWCOUNT
	
  DELETE o
  FROM #oper o
  INNER JOIN  @batch b
  ON ',STRING_AGG(CONCAT('o.[',c.columnName,N'] = b.[',c.columnName,N']'),N' AND '),N'
  
  DELETE FROM @batch

  IF (@ROWS=0)
    SELECT @ROWS=COUNT(*) FROM #oper WHERE __oper__=''INS''

END

DROP TABLE #oper
')
FROM #tgtColumns c
WHERE c.is_primary_key=1

DROP TABLE #tgtColumns

--PRINT @operSql
--PRINT @delSql
--PRINT @updSql
--PRINT @identityInsertOn
--PRINT @identityInsertOff
--PRINT @insSql

DECLARE @sql NVARCHAR(Max) = CONCAT(@operSql, @delSql, @identityInserton, @updSql, @insSql, @identityInsertOff)
EXEC sp_sqlexec @sql

END

OK, this is a large thing, but the principles used are simple:

  • first we create a table containing information about the columns of the tables: schema, table, column name, whether the column is primary key or computed, and the type name. The schema and table name are not used, but may be useful for debugging. Note that this SP doesn't check the tables have the same number and type of columns. That's on you to ensure.
  • using the column information we create four strings that will contain the SQL for the following operations:
    • create an "operations table"
    • delete rows that are not needed
    • update rows that need to be updated
    • insert rows that are missing
  • there are four strings mostly for debugging purposes to keep them smaller than the 8000 characters that Microsoft SQL Server Management Studio can print at a time, but they are concatenated and executed as one.
  • implementation details:
    • we use FOR XML PATH('Row'), ELEMENTS XSINIL to generate a string with all the data in each row, so we don't have to compare rows column by column. We could have made this work with comparisons, but the code would have been bulky and ugly when comparing for NULL or for having values. ELEMENTS XSINIL will ensure that there is a difference between empty space and NULL.
    • a FULL OUTER JOIN is used to find (based on the primary key columns of the destination table) if rows need to be either deleted, updated or inserted. That operation is specified in the __oper__ column
    • the operations table is thus created, containing the primary key values and the operation required with two indexes: one on the primary keys and one on the operation. These indexes are not really that relevant, so one could choose to remove them.
    • a @batch table variable is used with a PRIMARY KEY on the primary key columns of the destination table, for performance reasons
    • the batching is done via a DELETE... OUTPUT operation. We delete the 4900 rows we process and we output them in the @batch table
    • for each segment we either: delete destination rows with the same primary keys for 'DEL', update destination rows with the same primary keys for 'UPD' and insert source rows with the same primary keys for 'INS'
    • before we update, we set IDENTITY_INSERT to ON and at the end to OFF, if any identity columns
    • if some rows were affected by the operation, then we continue in the same segment. If not, then we look in the operations table to see if there are still rows to be processed. No rows may be affected while there are still rows to be processed due to the locking avoidance hints
  • improvements to avoid locking:
    • when we generate the operations table we use NOLOCK, which reads uncommitted values ignoring locks. This may not be what you want, but if the source table is locked for whatever reason, this ensures the merge operation is not blocked
    • when we process the batches we use READPAST, which ignores locked destination rows. This ensures that rows that can be updated will be, while the others can be done later, meaning that if you have 1 locked row, the merge operation will go around it, then wait until it is unlocked to update or delete it.

If you want to see what the generated SQL looks like, uncomment the PRINT lines and comment the EXEC one.

Now, I just wrote this stored procedure, so I may have missed some cases. Let me know if you find a situation where this doesn't work as expected. 

Hope it helps!

and has 0 comments

What is the structure of a table created from another via SELECT * INTO [Second] FROM [First] ?

A simple question, indeed, a basic one, but one that I have never asked myself until today. I honestly believed that the result is a generic table containing the same column names and types and their values and nothing else.

The answer, though, is strange. If the original column is NOT NULL, the resulting column will also be NOT NULL. If the original column has IDENTITY, the column in the second table will also have IDENTITY. And, as you know, you can't add or remove IDENTITY from existing columns without dropping and adding them back. The DEFAULT value, though, is not transferred.

The Microsoft reference page says: 

The format of new_table is determined by evaluating the expressions in the select list. The columns in new_table are created in the order specified by the select list. Each column in new_table has the same name, data type, nullability, and value as the corresponding expression in the select list.

...

When an existing identity column is selected into a new table, the new column inherits the IDENTITY property, unless one of the following conditions is true:

  • The SELECT statement contains a join.
  • Multiple SELECT statements are joined by using UNION.
  • The identity column is listed more than one time in the select list.
  • The identity column is part of an expression.
  • The identity column is from a remote data source.

...

Indexes, constraints, and triggers defined in the source table are not transferred to the new table, nor can they be specified in the SELECT...INTO statement. If these objects are required, you can create them after executing the SELECT...INTO statement.

...

Specifying an ORDER BY clause does not guarantee the rows are inserted in the specified order.

...

When a computed column is included in the select list, the corresponding column in the new table is not a computed column. The values in the new column are the values that were computed at the time SELECT...INTO was executed.

So, my assumptions were wrong, but still what do you do when you want to achieve exactly that: create a new schema-less table with the same columns in terms of name and data type and value only? It's a bit ugly, but I have not found a better alternative. Just UNION ALL with a SELECT that has the same number of (nullable) columns and no rows. like this:

SELECT *
  INTO [Second]
FROM [First]
  UNION ALL
(SELECT NULL, NULL,NULL,NULL WHERE 1=0)

Assuming that the First table had four columns, this will result in a table having the same column names, types and values as the original table, but all columns will be nullable and with no identity.

Hope it helps!

and has 1 comment

I have been maintaining a Chrome browser extension I wrote for more than a year now and I always get the occasional user asking me if I can make it work with Firefox. And until now I said no, because I use the "world" feature in manifest.json. But I was wrong.

You see, Google is aggressively pushing for manifest version 3, discontinuing support for browser extensions using v2. So my extension was written from the beginning with version 3. In order to load the scripts and CSS files that I needed to run in the context of the page, I used the world:"MAIN" feature. When I tried it with v2, Chrome immediately told me "The 'world' property is restricted to extensions with 'manifest_version' set to 3 or higher." (Or higher. lol) So when I looked for how to use world in Firefox manifest v3 I got a lot of documentation about Xray Vision and how they absolutely refuse to implement the world feature.

I googled, I scoured the web, I read all of Stack Overflow, I even tried LLMs like ChatGPT of Gemini to hilarious results, like Gemini accusing me I want to circumvent the security of the browser. No. There is no simple way of doing the same thing in Manifest v3 on Firefox. Case closed, right? I mean, who even uses Firefox? Less than 2.5% of people on the Internet. And it's a shit browser.

But no, here comes a guy and tells me that he made it work. HOW?! By now you probably guessed it, I left enough hints after all. You just have to use manifest version 2! 'world' is only restricted for version 3 in Chrome browsers!!!

Although now that I am writing this post I see stuff like Manifest V3 updates landed in Firefox 128 so maybe it something that works only recently (the article is from July 2024). Hmm..

I guess there is a silver lining to the fact I refused users for a year now, because I had a year in which I didn't have to deal with the buggy debugger (heh!) in Firefox.

Anyway, if you have the same problem, that's your solution: make a v2 manifest for Firefox and a v3 manifest for Chromium browsers. Or use it in v3 as well, because apparently Firefox changed their minds. And I wrote this blog with such glee that I would help people that had the same problem as me. Ugh!

NO! I will not make it work for Safari! Eat a dick!

and has 0 comments

  Just a few days ago I was writing on how important it is to tell Entity Framework what SQL type to use in order to avoid costly conversions. In fact, it wasn't so much an EF issue as it was an SQL one. Converting even character types to character types or changing collation was surprisingly expensive.  In this post I will show you how important it is to choose the right type for your querying columns, especially the primary key. 

  First imagine this scenario: you get some data from an outside source, rows and rows of it, and you have to store them and query them in SQL. The value that uniquely identifies a row is a small string, maybe 50 characters long. How do you proceed?

  My first naive solution was the most obvious one: just create a table that has a column for each value in the rows and put the primary key on the identifying one. But this leads to immediate performance losses:

  • by default, a primary key is a clustered index - text is not sequential, so at every insert the database engine will physically move huge swaths of data in order to place the rows in the alphabetical order of their identifiers
  • a primary key is a unique index - meaning text will have to get compared to other text in order to determine uniqueness, which is slow
  • by default, SQL is case insensitive - meaning that all text comparisons will have to be made taking into account capitalization and accents
  • 50 characters is a lot - even without Unicode support, it's 50 bytes, which is 12 times more than an integer, meaning the primary key index will be large; and slow

  "But!", you will undoubtedly say, if you put the primary key on some other column, you will still have to create a unique index on the identifier. Isn't this just pushing the problem farther down the road? The size and speed limitations will be the same. And primary keys are clustered only by default, but they can be declared as not clustered. And SQL doesn't need to be case insensitive, all you have to do is change the collation of the column to be binary and it will be compared faster. Wouldn't that solve the problem?

  No. In fact, my final solution which worked five times faster, did not have an index on the identifier column AT ALL. Incidentally, I did end up changing the collation, but only because the idiots sending me the data were doing it case sensitive.

  Without further ado, here is what I did:

  • an INT column with IDENTITY(1,1) as the primary key - which ensures a fast insertion due to the sequential nature of the value, fast query speed and low usage of disk space for the index
  • an INT column holding the checksum of the identifier - which when indexed, is fast to query and doesn't use a lot of disk space for the index

   So how do I query on the identifier? Simple: I calculate the checksum of the string and then I look it up in the database - which uses the index to locate the few strings that have the same checksum, then just finds the right one by enumerating through them. I query on the checksum column AND the text identifier. And there is an added bonus: I only need to do this once. If I need the record from the DB again, I query it directly through the integer primary key.

  Entity Framework has this automatic memory cache so when I am querying on the database entity using a business model - as good separation of concerns practice would dictate - it gets it really fast from memory. Because the memory cache also uses just the int to identify an entity, which means double the benefits!

  The eagle eyed reader will have noticed that I am not using a unique index on the identifier, so technically I could create multiple rows with the same one. However, my application is always looking for the existing record first. But if you really worry about data consistency, the index on the checksum column can be replaced with a unique index on the checksum and identifier column. It will take more space, but it will be just as fast.

  Another thing that you may have noticed is that I use a code checksum, not the database provided functions to achieve the same. At first glance, it's an instant win: just create a persisted computed column that calculates the checksum or binary checksum of the identifier column. However, this would be weird when having to query, since you would have to craft a stored procedure or a custom SQL command to get the identifier and query on its checksum. In my case I just calculate a checksum - and not use the lazy string.GethashCode function which may be subject to change and it's already different between 32 and 64 bit systems.

  Of course, if you want your text columns to be case and/or accent insensitive, you will have to store the hash code of the lowercase and unaccented string or use an implementation that is case and accent insensitive. This may not be trivial.

  Further tests showed that just using a non clustered index on the identifier column, even a unique one, was just slightly slower, maybe 5%. However, the space taken by indexes increased by 20%. So I might understand why you would find it a bit off putting and skip the checksum part.

  Hope this helps!

  P.S. Why did this solution provide such a huge performance gain? Obviously the SQL team would have implemented a sort of checksum for their text index, this should have been working natively and faster than any possible implementation I could make. Well, I don't know the answer. In fact, this all could be some quirk of Entity Framework and the SQL queries would not be optimizable to such a degree. I will attempt to test that using purely SQL commands. But meanwhile, all the points I made above are valid and with a little more work you can have a lot more control on how the system works.

and has 0 comments

  I've built an application and, like any lazy dev out there, I focused on the business logic, the project structure, the readability, comments, the dependency injection, the unit tests, you know... the code. My preference is to start from top to bottom, so I create more and more detailed implementations of interfaces while going down to the metal. The bottom of this chain is the repository, that class which handles database access, and I've spent little to understand or optimize that code. I mean, it's DB access, you read or you write stuff, how difficult can it be?

  When it was time to actually test it, the performance of the application was unexpectedly bad. I profiled it and I was getting reasonable percentages for different types of code, but it was all taking too long. And suddenly my colleague says "well, I tried a few things and now it works twice as fast". Excuse me?! You did WHAT?! I have been trying a few things too, and managed to do diddly squat! Give me that PR to see what you did! And... it was nothing I could see.

  He didn't change the code, he just added or altered the attributes decorating the properties of models. That pissed me off, because I had previously gone to the generated SQL with the SQL Profiler and it was all OK. So I executed my code and his code and recorded the SQL that came out:

  • was it the lazy loading? Nope. The number of instructions and their order was exactly the same
  • was it the explicit declaration of the names of indexes and foreign keys? Nope. Removing those didn't affect performance.
  • was it the ChangeTracker.LazyLoadingEnabled=false thing? Nope, I wasn't using child entities in a way that could be affected.
  • was there some other structure of the generated SQL? No. It was exactly the same SQL! Just my code was using thousands of CPU units and his was using none.
  • was it magic? Probably, because it made no sense whatsoever! Except...

Entity Framework generates simple SQL queries, but it doesn't execute them as you and I would. It constructs a string, then uses sp_executesql to run it. Something like this:

exec sp_executesql N'SELECT TOP(1) [p].[ID], [p].[TXT], [p].[LUP_TS]

FROM [sch].[table] AS [p]

WHERE [p].[ID] = @__p_0',N'@__p_0 nvarchar(64)',@__p_0='xxxx'

Do you see it? I didn't until I started to compare the same SQL in the two versions. And it was the type of the parameters! Note that the aptly named parameter @__p_0 is an NVARCHAR. The actual column in the database was VARCHAR! Meaning that the code above was unnecessarily always converting values in order to compare them. The waste of resources was staggering!

How do you declare the exact database type of your columns? Multiple ways. In my case there were three different problems:

  • no Unicode(false) attribute on the string columns - meaning EF expected the columns to be NVARCHAR
  • no Typename parameter in the Column attribute where the columns were NTEXT - meaning EF expected them to be NVARCHAR(Max)
    • I guess one could skip the Unicode thing and instead just specify the type name, but I haven't tested it
  • using MaxLength instead of StringLength - because even if their descriptions are very similar and MaxLength sounds like applying in more cases, it's StringLength that EF wants.

From 40-50ms per processing loop, it dropped to 21ms just by fixing these.

Long story short: parametrized SQL executed with sp_executesql hides a possible performance issue if the columns that you compare or extract have slightly different types than the one of the parameters.

Go figure. I hate Entity Framework!

and has 0 comments

Intro

  This post is about the System.InvalidOperationException: This SqlTransaction has completed; it is no longer usable. which may be because you shared your SqlConnection or you tried to SaveChanges twice and all of the other issues that you can google for. I was not so lucky. I spent a day and a half to understand what's going on and only with a help of another dev did I get close to the issue.

TL;DR;

I used a column with identity generation, but it wasn't also a primary key and EF sucks.

Details

  Imagine my scenario first: I wanted to use a database to assign a unique integer to a string. I was first searching for the entry in the DB and, if not found, I would just insert a new one. The SQL Server IDENTITY(1,1) setting would insure I got a new unique value for the inserted row. So the table would look like this:

CREATE TABLE STR_ID (
  STR NVARCHAR(64) PRIMARY KEY,
  ID INT IDENTITY(1,1)
}

Nothing fancy about this. Now for the C# part, using Entity Framework Core 6.

I created an entity class for it:

[Table("STR_ID")]
public class StrId {

  [Column("STR")]
  [Key]
  public string Text { get; set; }

  [Column("ID")]
  [DatabaseGenerated(DatabaseGeneratedOption.Identity)]
  public int Id { get; set; }

}

And then I proceeded to test it in the following way:

  • create a DbContext instance
  • search for a value by STR/Text in the proper DbSet
  • if it doesn't exist, insert a new row and SaveChanges
  • retrieve the generated id
  • dispose the context

I also ran this 20 times in parallel (well, as Tasks - a minor distinction, but it was using the thread pool).

The result was underwhelming. It would fail EVERY TIME, with either an exception about deadlocks or 

System.InvalidOperationException: This SqlTransaction has completed; it is no longer usable.
   at Microsoft.Data.SqlClient.SqlTransaction.ZombieCheck()
   at Microsoft.Data.SqlClient.SqlTransaction.Commit()

I did what every sane developer would do in this situation and bought myself a shotgun (we all know it's the most effective against zombies) then googled for other people having this issue. I mean, it would be common, right? You do some EF stuff in parallel and you get some errors.

No. This is happening in a parallelism scenario, but that's not the cause. Also, it's not about transactions. EF will wrap SaveChanges operations in a transaction and that is causing the error, but the transaction being completed is the issue and no, it's not your code!

I tried everything I could think of. I disabled the EF transaction and made my own, using all types of IsolationLevel, I tried EnableRetryOnFailure with hilarious results (I was monitoring the values inserted in the database with NOLOCK and they were going to 20, then back to 10, then 15, then back to 1 and it was taking ages trying to retry operations that apparently had dependencies to each other, only to almost all to fail after a long time). I even disabled connection pooling, which probably works, but would have made everything slow.

Solution

While I can't say what EXACTLY caused the problem (I would have to look into the Microsoft code and I don't feel like it now), the solution was ridiculously simple: just make the IDENTITY column a primary key instead:

CREATE TABLE STR_ID (
  ID INT PRIMARY KEY IDENTITY(1,1),
  STR NVARCHAR(64)
}

-- because this is what I am searching for
CREATE UNIQUE INDEX IX_STR_ID_STR ON STR_ID(STR) 
[Table("STR_ID")]
public class StrId {

  [Column("ID")]
  [Key]
  public int Id { get; set; }

  [Column("STR")]
  public string Text { get; set; }

}

I was about to use IsolationLevel.ReadUncommitted for the select or just set AutoTransactionsEnabled to false (which also would have solved the problem), when the other guy suggested I would use this solution. And I refused! It was dumb! Why the hell would that work? You dummy! And of course it worked. Why? Donno! The magical thinking in the design of EF strikes again and I am the dummy.

Conclusion

What happened is probably related to deadlocks, more specifically multiple threads trying to read/write/read again from a table and getting in each other's way. It probably has something to do with how IDENTITY columns need to lock the entire table, even if no one reads that row! But what it is certain to be is a bug: the database functionality for a primary key identity column and a unique indexed identity column is identical! And yet Entity Framework handles them very differently.

So, in conclusion:

  • yay! finally a technical post
  • this had nothing to do with how DbContexts get disposed (since in my actual scenario I was getting this from dependency injection and so I lost hours ruling that out)
  • the error about transactions was misleading, since the issue was what closed the transaction inside the Microsoft code not whatever you did
  • the advice of some of the AggregateExceptions up the stream (An exception has been raised that is likely due to a transient failure. Consider enabling transient error resiliency by adding 'EnableRetryOnFailure' to the 'UseSqlServer' call.) was even more misleading
  • the EF support for IDENTITY columns - well, it needs it because then how would it know not to attempt to save values in those columns - is also misleading, because it doesn't mean it's good support
  • while parallel access to the DB made the problem visible, it has little to do with parallelism 
  • EF knows how to handle PRIMARY KEYs so that's the solution
  • EF sucks!

I really hope this saves time for people in the same situation!

and has 2 comments

Happy Birthday, LiChess Tools!

It was one year ago that LiChess Tools was first published on GitHub. It was like the birth of a child, having spent a few weeks in gestation as an extension I would only use for myself. It was doing just the simplest of things at the time:

  • opening friends box automatically on page load, so that you see if you have any friends online
  • making sound alerts when friends started playing, so you can watch your favorite chess players the moment they start a game
  • pressing Shift-Right to choose a random variation from current position in analysis or study
  • sticky Preview mode in studies, so that you can move from chapter to interactive chapter and keep playing without seeing the move list
  • setting a minimum engine depth, so that the local engine would run automatically if a lower cloud depth was displayed

It had the Preferences in the extension popup, because they were so few features. The code was so awful that I didn't dare tell anyone about it.

Now this has become a behemoth with close to 100 different tools and bright prospects for the future.

I would like to thank the community, such as it is, because even if only one person was giving me feedback in a month, it could have happened when I was feeling low or stressed or unmotivated and it would perk me up immediately. Thank you a lot, guys!

For some weird reason (all passionate devs are weird) there was nothing more motivating than some kid wanting a feature, first thinking it was impossible, then getting the nagging feeling that I should think about it more, then finding a brilliant lateral solution, implementing it, improving on it, then delivering everything within the hour only to get a bored "thanks" at the end. But that thanks was all I needed to carry on. Occasionally I get thankful notes from people and it makes my day.

Right now LiChess Tools has 2500 daily users and 26 ratings that average to 4.8 stars. It's not the quantity, but the quality, though. The extension is focused on chess analysis and ease of learning. It's basically a pro tool, aimed at chess enthusiasts, coaches, schools and chess professionals. With such a scope, 2500 users is huge! And we'll get even higher.

At the time of this writing, plans are in motion to use the OBS integration feature of LiChess Tools for the official Lichess Sharjah Masters broadcast on the 14th of May, presented by WIM Irene Kharisma Sukandar. Oooh, I hope it doesn't break midway 😱

[youtube:kslb3y4W7RM]

But there is more! I am working with the Lichess devs to streamline some of the more hackish features of the extension so that it can be used en masse without bothering the Lichess servers. I've received some suggestions from GMs and IMs and chess coaches that I will implement in LiChess Tools and I will support a plan to update the chess openings list in Lichess (as well as in Lichess Tools).

So there are some great opportunities open to the extension in the near future and hopefully they will make this blossom into something even more special!

The next major version (3.*) will probably restructure the features into more mature tools, focus on performance and adding more "epic" features, like:

  • full Client Side Analysis - including brilliant/good/best move detection ideas, statistics and charts
  • a more complete and user friendly Explorer Practice module
  • Chessable-like interface for Studies and spaced repetition features

There is also time for a rebranding. I am tired of people thinking I am talking about the Tools menu in Lichess. Right now the best idea I have is Prometheus for Lichess. I just hope Thibault is not going to nail me to a mountain and sic the Lichess owl on my liver. Perhaps you guys can come with better ideas.

Rebranding doesn't come with corporate goals and premium tiers, though. LiChess Tools will always be free, regardless of its name, so don't worry.

So, let's celebrate by singing along with the official LiChess Tools theme and hope for an even more awesome year!

It's made with AI, so it's cool by default 😁

Enjoy chess, Lichess and LiChess Tools!

P.S. Bring me your stories, people! I want to know how you use the extension. Join the LiChess Tools users team and share your experience with all of us.

and has 0 comments

You are writing some Javascript code in the browser and you create classes, then you create some methods, which you see have some issue with the "this" keyword by default if you use the standard method declaration so you end up doing something like this:

// instead of myMethod() { ... }
myMethod=()=>{
  const myClass=this;
}

And this (pardon the pun) works here. Imagine my surprise when I did something that seemed identical:

// instead of myMethod() { ... }
myMethod=()=>{
  debugger;
}

And then I tried to see what "this" was in the Chrome Developer Tools. It was "undefined". Why?!

Long story short, I then tried this (heh!) and it worked:

// instead of myMethod() { ... }
myMethod=()=>{
  const myClass=this;
  debugger;
}

The moral of the story is that "this" is not declared unless used in code. Probably a browser optimization. Hope it saves you the ten minutes it took me to understand what was going on, after I've started doubting my entire JavaScript expertise and the entire design of my app.

and has 0 comments

As you know, LiChess Tools is my own Chromium browser extension for lichess.org, adding a lot of stuff to it. Recently, from version 2.2.0 on, I've added a new feature called Show Pawn Structures. This post explains in detail what it is, what it does and how it does it.

Pawn Structures

What is a pawn structure? The configuration of pawns on the chessboard. Because pawns are the least mobile of the chess pieces, the pawn structure is relatively static and thus plays a large role in determining the strategic character of the position. It is a powerful chess concept that is mostly ignored on amateur level and I've met 2000+ rated players who didn't know what that was. Not that I know, either, which is why I got so excited to build this feature because it would further chess understanding and learning. With different structures come medium term plans, so instead of having the opaque engine recommendations of making one move or another, you will have a general idea on where to take the game to.

The above is the chess definition of the concept, though. In order to work with it in an algorithm it has to be clearly defined. The difficulty here lies in the fact that while the pawn structure is "relatively static" its meaning is not. While you will be shown a specific pawn configuration in the context of a named structure, it would be implied that other similar configurations also belong. That similarity being not precise, but something nebulous related to the general ideas and themes that are made possible by the structure.

Feature requirements

The purpose of the feature is to determine the pawn structure of a position in either game analysis, analysis board, studies, TV games and mini-games (the things that appear when you hover on a playing user link or in the Current Games section), then display it, similar to the Show Opening feature. The reasoning here is that one can learn to classify positions and thus know the general plans that apply in the situation.

Technical details

There is a list of pawn structures that LiChess Tools supports. The list is at the end of this post. In order to the structure I created a textual representation of them, that looks something like this: 312100TX 0X0210 2020 XXLXXX XXXXXX XXXX. A bit daunting, but the most important part is the first group of characters: 312100TX.

The idea is that the first characters are the most significant, so similar pawn structures would start with the same letters and digits, even if they diverge later on. Based on the structures detailed by various books on the matter, I've considered that the position of the d-pawn is the most significant, followed by the e and c pawns, then the pawn majority on the White and Black sides, followed by the other pawns: f,b,g,a,h. The final part is doubled or triple pawns, which most of the time is irrelevant.

So let's get back to the complicated string above: 312100TX 0X0210 2020 XXLXXX XXXXXX XXXX (we will assume White board orientation)

  • 312 - my pawn position on the d,e,c files: d5 (3 squares forward), e3 (1 square), c4 (2 squares) - the possible characters for this group are X (missing pawn), 0 (unmoved pawn), 1,2,3 (squares the pawn is forward from its starting position)
  • 100 - their pawn position on the d,e,c files: d6, e7, c7
  • TX - the majority on the queenside and kingside: T (they) have majority on the queenside, and equality on the kingside - the possible characters for this group are M (me), T (them) or X (neither)
  • 0X0 - my pawn position for the f,b,g files
  • 210 - there pawn position for the f,b,g files
  • 20 - my pawn position for the a,h files
  • 20 - my pawn position for the a,h files
  • XXLXXX XXXXXX XXXX - similar to above groups, doubled or tripled pawns - X for no, L for yes

Most pawn structures are exclusively about the position of the c,d and e file pawns, therefore the first group of characters will be the most determining one. This may change in the future, perhaps, as better chess players than me will point out issues with this system, but for the moment this is how it works.

Based on this string we can compare the current pawn position to the one of the named pawn structures. There are three options in the Preferences for this feature:

  • Enabled - if this feature is on or off
  • Only named structures - will only display the named structures if enabled. If not, then the name will be displayed as the most significant group of characters in the structure representation. On mouseover, the title will show the entire thing as well as the most similar named structure found.
  • Fuzzy - if this feature is on, then a structure will be considered named if 90% similar to the standard one.

The feature will invert the structure and look for a name there if not found for your own orientation. If a name is found there, "(R)" will be added to the name. 

Note that the most named structures are represented by the most significant group only, and only several of them by the first two or three groups of characters. The rest is ignored.

Similarity

Now, how do we compute the similarity? It's a work in progress, but at the moment it works like this:

  • the number of common characters on the same position from the beginning of the text are counted as 1
  • the number of other common characters on the same position (so after any position where the characters were different) are counted as 0.8
  • the number of positions where the current structure has pawns on squares that may allow moving them on the named structure square for that position (so where the value is numerical and smaller than the numerical value of the named structure) are counted as 0.2
  • the percentage of the resulting sum from the characters counted is returned as the result

Example: Carlsbad vs Slav
21X2X0TM
21XX11TX
11100280 (8 denotes 0.8 and 2 denotes 0.2 here) = 4/8 = 50%

It may be that this is not the best way and it might be changed in the future.

List of structures

A small note on naming: different sources name these differently or don't recognize them as structures at all. I did what I could to give a short identifiable name to each position, but I think sooner or later I will have to restrict the number of names, rather than increase it. We'll see.

So here is the list of pawn structures recognized by LiChess Tools (v2.2.3):

The links above are also used in LiChess Tools and are mostly from Wikipedia, but also some approximations or just other random sites because there are no links for many of the Flores structures. I would be very happy if someone would help me clean these up.

Hope this explains everything. Enjoy!

and has 0 comments

Note: This article is about Chromium based browsers.

Remember the days when computers were configured to present the filesystem to the network by default in read/write mode? Those were the days. Today, though, everything is configured for security and browsers are no exceptions. One thing that annoyed me yesterday was CSP (Content Security Policy) which disallowed me to fetch from a web site information from another web site which was happy to provide it. The culprit was a meta tag that looks like this:
<meta http-equiv="Content-Security-Policy" content="...">

The content was configuring default-src, connect-src, style-src, frame-src, worker-src, img-src, script-src, font-src! Everything. But I wasn't looking for a hack to disable CSP (well I was, but that's another story), I just wanted to test that, given a friendly CSP, I could connect to a specific web site and get the data that I wanted and do something with it. Surely in the developer tools of the browser I would find something that would allow me to temporarily disable CSP. No such luck!

Then I looked on the Internet to see what people were saying. All were complaining about "Refused to connect to [url] because it violates the following Content Security Policy directive..." and how it annoyed them, but there was no real solution. Here is what I found:

  • browser extensions to remove the CSP header
    • I assume this works, but it wasn't my case
  • browser extensions to remove content from the page from the Developer Tools
    • I tried one, but when it changed the content now the browser was crashing with an ugly Aw, snap! page with a Status_Access_Violation status
  • I tried ticking the web site's settings for Insecure content
    • How naïve to think that it would allow loading of insecure content
  • I tried browser command line flags and experimental flags
    • nothing worked

I was contemplating hacking the browser somehow when I stumbled upon this gem: Override files and HTTP response headers locally.

It is almost exactly what I was looking for, only it doesn't replace content with regular expressions but saves the entire content of a URL on the local drive and serves it from there, modified in whatever way you want. So if you want to alter a server rendered page you're out of luck.

How did I use it to remove the CSP? I went to sources, I configured the local overrides and I then edited the page (in the Sources panel) and simply deleted the annoying meta tag. Now it worked.

Hope it helps!

and has 0 comments

Some time ago I wrote a post about the difference between temporary tables (#something) and table variables (@something) which concluded that before SQL Server 2019 table variables sucked. But I didn't give any real example. I made it very technical and clinical, perhaps creating the wrong impression that it's a matter of minute optimizations and not something somebody should know.

So gather 'round, children, and let me tell you the tale of the dreaded table variable! How a query that timed out after an hour was run in just 3 seconds with a tiny little fix. The server version was... 2017 (fade to dream sequence)

First, the setup: I needed to delete rows from a large table (30+ million rows) which did not have an associated account in a calculated list of about 20 thousand. Simple right?

DECLARE @Accounts TABLE(ACC_ID CHAR(5))

INSERT INTO @Accounts
SELECT ... whatever
      
SELECT TOP 1 *
FROM MyTable t
LEFT OUTER JOIN @Accounts a
ON t.ACC_ID=a.ACC_ID
WHERE a.ACC_ID IS NULL

Just get me that first orphaned record please... ... ... 40 minutes later... WTF?

As I was saying in the previous post, table variables do not have statistics and the engine assumes they have only one row. So what does the execution plan want? To go through all of the 30+ million rows and then search them into the accounts table. Only the accounts table has 20 thousand rows, too. And it takes forever!

Mind that I've simplified the case here. Table variables do allow indexes and primary keys. I tried that. No effect at all!

Let's change the query then:

CREATE TABLE #Accounts(ACC_ID CHAR(5))

INSERT INTO #Accounts
SELECT ... whatever
      
SELECT TOP 1 *
FROM MyTable t
LEFT OUTER JOIN #Accounts a
ON t.ACC_ID=a.ACC_ID
WHERE a.ACC_ID IS NULL

DROP TABLE #Accounts

Just get me... wait, what? The query ended in 3 seconds. Suddenly, knowing the number and structure of the rows in the table led to the correct execution plan.

But can't I use table variables somehow? You can, but you have to force the engine to do it your way. First of all, you use OPTION (FORCE ORDER) which will keep the tables in the order you declared them. And then you have to reverse the JOIN so that @Accounts is the first table, but the effect is the same.

DECLARE @Accounts TABLE(ACC_ID CHAR(5))

INSERT INTO @Accounts
SELECT ... whatever
      
SELECT TOP 1 *
FROM @Accounts a
RIGHT OUTER JOIN MyTable t
ON t.ACC_ID=a.ACC_ID
WHERE a.ACC_ID IS NULL
OPTION (FORCE ORDER)

Back to three seconds. Ugh!

Now, I will probably use a temporary table, because forcing the way the SQL engine interprets your queries is almost always a bad idea and it makes the code harder to read, too.

Hope it helps!

and has 4 comments

LiChess Tools is a Chromium-based browser extension (meaning Chrome, Brave, Edge, Kiwi - which also works on mobiles - and others, as long as they have a Chromium version equal or higher to 111) and recently also on Firefox and Firefox Android. The extension extends the functionalities of the lichess.org web site. This page is the user manual for the extension, cataloguing its current features and explaining how to use them.

All of the features below can be individually turned on or off from the Preferences menu, the LiChess Tools menu entry, or - if not logged in - from the LiChess Tools menu on the top right. Note that many of the options will be hidden until you toggle Advanced Preferences - these features will be marked with Advanced in the category section.

Preference values can be backed up as a downloaded file and restored from it. You can also reset preferences to default or turned them all off, then turn on the ones you are interested in. The blueish border around some preference values means they are part of the default configuration. A sepia coloring of a preference and a WIP added to the title means the feature is "Work in progress" and results are not guaranteed.

Note: the version number below is often the one if development, so there might be differences between what you have and what is listed here.

Here are the available features in LiChess Tools v2.3.132 in the order of decreasing approximate importance:

Crowdin translation

category: Languages
values: yes / no - defaults to yes

Crowdin is a community driven translation service that can help handle text translation. LiChess Tools has integrated translation from there (see https://crowdin.com/project/lichess-tools), but the only languages I can be sure of are English and Romanian. From the other 10 supported languages, many of them are translated by machines, so they are probably really bad. If you have your Lichess in your language, but you can't stand the bad translation in the LiChess Tools strings, you have two options:

  • turn off Crowdin translation and the extension text will now be in English
  • go to Crowdin and help fix the translation

Obviously, I recommend the latter.

LiChess Tools team

category: Community
values: Hide forum entry / Forum entry last / No notifications - defaults to none
Needs log in

This will add a new forum entry in the forum list from where you can join the LiChess Tools users team or, once joined, you can participate in the extension community.

Note: you will be shown a notification to join the team. You can disable that just by visiting the team page, then it will not appear again on your device. The "No notifications" setting below does the same thing.

Options:

  • Hide forum entry - If set, this will remove the forum entry suggesting you join the LiChess Tools users team (or, if you joined already, link to it)
  • Forum entry last - If set, the forum entry will be placed last in the main list of forums (after Off-topic forum) 
  • No notifications - If set, this feature will never send notifications. If not set (the default value), it will add a notification to join the team. Just visiting the team page will also disable the notification, whether you join or not. This will also block any future team notifications I might choose to send into the future.

Being part of the team will be advertised on your public profile as well as on your friend's timelines. That is great for LiChess Tools, so if you want to support the extension, please consider joining.

I don't plan to spam anybody in the team, just use it to announce various changes to the extension. Which are a lot and happen all the time and might be construed as spam... OK, I will not spam you much. And also I would like to hear from you, maybe even (gasp!) form a community that will take this farther than my meager imagination could take it.

While in the team, feel free to talk to the other users, ask for features, complain, all the jazz. Use the team's forum as one would use the official Lichess Feedback forum. Only less divisive :). And don't worry, the team is called L1Chess Tools because Lichess forbids the text "lichess" in the team's name, not because I plan a L2Chess Tools team or whatever.

Although that is an idea...

Extended interactive lessons

category: Study
values: Play all variations / Show final score / Always show score / Play again from where you entered Preview mode / Fast interaction / Give up button - defaults to Play all variations, Show final score

This feature may be the flagship of the extension, as it allows you to play an Interactive lesson chapter by going through all variations instead of just the mainline. The default lichess behavior is to only follow the mainline, considering all sidelines as bad. With the new behavior all moves in the PGN tree are good, while the ones that are not in the PGN are bad. You can mark the bad variations with glyphs (good move, mistake, blunder, etc.), for example, to indicate if a move is bad or good, but you can continue playing it to see why. 

There is a mechanism to choose what the computer will play next as your opponent. It uses the number of branches in 8 ply (4 chess moves for both players) to determine which is more probable. This value can be changed (see Next move probability depth feature) For example if you have something like 1. e4 (1. d4 d5 2. Nc3 (2. Nf3)) 1... e5 2. Nf3, the mechanism will see that there are two possible branches for d4 as opposed to one for e4, so it will choose d4 66% of the time. The probability can be changed manually by adding a comment in the move with the format prc:<number>, where the number is a percentage between 1 and 100. (ex: prc:90) Also, if set in the Transpositions behavior feature, the moves can be picked from moves following transposing positions. 

When hints are not provided by the study creator, one will be generated to show how many possible moves are available. The hint link will show a number of possible moves, if higher than 1.

There is also a system to compute accuracy when playing an interactive lesson by counting the good and the bad moves. This will be displayed as a percentage during or at the end of the lesson and can be turned configured by option.

The lichess UI remains the same, therefore the area in which to explain why any other move is wrong only appears on mainline moves. That is why this feature also adds a context menu item (Explain why other moves are wrong) for nodes in interactive lessons to enable editing that information. As usual, when a "bad" move is made (meaning a move not in the PGN), the text to be displayed to the user will be taken from the first next move from the current position. Therefore in order to make this work you have to right-click that node and explain why bad moves are wrong. This might be a bit counterintuitive, as you may have several moves following the current position, but the explanation has to be set to the first next move.

This feature also adds a header like Extended Interactive Lesson over the edit controls specific to interactive lessons. They usually take a lot of space, so clicking on this header will hide/show them. This allows people to edit Interactive Lesson chapters just as comfortable as normal ones.

Options:

  • Play all variations - enables the Extended mode for interactive lessons, allowing you to play all variations with the computer picking random moves from the ones available in the PGN
  • Show final score - shows the score at the end of the lesson. One can disable this to reduce frustration or for other reasons. Mouse over the score to see number of good and bad (or asking for solution) moves.
  • Always show score - shows the score at every step of the lesson. This is disabled by default.
  • Play again from where you entered Preview mode - this feature is disabled by default. It remembers the position where you entered Preview mode for an Interactive Lesson and, when the lesson ends, the Play again button will take you back to that position instead of the first move in the chapter. Obviously, this only works for creators or contributors to studies.
    • warning, if you are playing for completion, like having a good score or using the lesson flow features, this will probably interfere with that.
  • Fast interaction - this will remove the delay between moves, making things faster. Also, the Good/Retry large tiles will be replaced by small symbols on the comment bubble: green checkbox for good move and red x for bad move.
  • Give up button - this will render a "Give up" button next to the hint in interactive lessons. Pressing it will be equivalent to making a bad move and going directly to the end menu of the interactive lesson.

Some of these options are also available in the action menu (hamburger button) of the Analysis board when in a study chapter that is of type Interactive Lesson. This allows to quickly enable/disable just these relevant settings.

Notes:

  • one can play all variations of an interactive lesson with multiple branches only with the LiChess Tools extension installed, so if you create such a lesson only people with the extension can enjoy it. It would be nice to let everybody know that in a comment if you share it.
  • asking to show the next move will show arrows of all possible next moves. Be warned that pressing the button is seen by the scoring system as a fail.

Extended interactive lesson flow

category: Study - Advanced
values: Sequential / Spaced Repetition - defaults to none

Work in progress! - this means use at own risk.

This feature allows you to play Extended Interactive Lessons variation by variation. When any of these options are enabled, the chapters that had variations played will have a small progress indicator in the chapter item in the chapters list. When all variations have been played a reset prompt will appear. You can also manually reset the played variations from the chapter edit dialog for the studies you can modify and from the reset button on the chapter item for the ones you can't.

Options:

  • Sequential - this will force playing extended interactive lessons variation by variation, in order
    • if you make any mistakes, you will start over
    • if you don't play from the very beginning of the variation, it will not count
  • Spaced Repetition - this will force playing extended interactive lessons variation by variation, randomly, based on success or failure
    • if you finish a variation successfully, the time until you can play it again will double
    • if you finish it with mistakes, the time until you can play it again will become two days
    • if you don't play from the very beginning of the variation, it will not count  
  • Both - this will behave as Spaced Repetition, but will take variations in order
    • the distinction is small, but it matters to some players to get the first variation that can be played

More options - only meaningful in the context of either or both Sequential or Spaced Repetition being enabled:

  • Avoid lines marked as mistakes - It makes sure that lines that contain moves of yours marked as dubious, mistake or blunder are not counted.
  • Hint excluded moves - In some lessons it's hard to understand which line has been picked as "right". You can use show solution first and then play it, but this will also help. When you have multiple options from your position, but only one is being trained, the hint will show the moves to avoid. So something like "Only one accepted move. Avoid b4, Nf3".

Notes

  • The default replay interval is one day.
  • This is a work in progress. It might not work exactly as you expect it. Please report any issues to me.
  • The currently selected Extended Interactive Lesson settings will be displayed when playing an Extended Interactive Lesson.

Ctrl-right to play random next move from list

category: Analysis, Study
values: yes / no - defaults to yes

This feature will allow you to play a random move from the PGN by pressing Ctrl-right. There is a mechanism to choose what the computer will play next as your opponent. It uses the number of branches in 8 ply (4 chess moves for both players) to determine which is more probable. This value can be changed (see Next move probability depth feature) For example if you have something like 1. e4 (1. d4 d5 2. Nc3 (2. Nf3)) 1... e5 2. Nf3, the mechanism will see that there are two possible branches for d4 as opposed to one for e4, so it will choose d4 66% of the time. The probability can be changed manually by adding a comment in the move with the format prc:<number>, where the number is a percentage between 1 and 100 (ex: prc:90). Also, if set in the Transpositions behavior feature, the moves can be picked from moves following transposing positions. 

Ctrl-left works almost like the left key, going to the previous position, but in this case it will go to the position you came from, not from the previous position on the line you've jumped to.

Notes:

  • if set in Transpositions behavior, Ctrl-right may choose a move from another line, but made from the same position, which means that going "back" will go to the previous position in the current line, not the original one. Use Ctrl-Left to return to the position you jumped from. 
  • the keyboard shortcuts will be shown in the keyboard help popup if this feature is enabled (press Shift-? for the help popup - this is a native lichess feature)

Behavior of transpositions

category: Analysis/Study - Advanced
values: exclude if same line / don't add identical next moves / play moves from transpositions - defaults to none

This feature determines what a transposition is for other features that use transpositions. The options are:

  • exclude if same line - two identical positions in the same PGN will not be considered transpositions of one another if they are in the same line of moves (one came before the other) (see Highlight moves in analysis and Show next moves from transpositions)
  • don't add identical next moves - when showing next possible moves at the bottom of the move list, if adding them from transposing positions, don't add the same move twice (see Show next moves from transpositions)
  • play moves from transpositions - when picking a next move to play, use transposing positions as well (see Ctrl-right to play random next move from the list and Extended Interactive Lessons)
  • none - when none of the values above are selected, any two identical positions will be considered transpositions, the move list can show multiple identical moves if they are coming from different transpositions/lines and the extension will not pick next moves using transpositions.

Next move probability depth

category: Analysis/Study - Advanced
values: equal / one move / two moves / three moves / four moves / five moves - defaults to four moves (8 ply)

This configures the mechanism calculating the probability of a next move in the PGN (see Extended Interactive Lessons and Ctrl-right to play random next move from list). The default value goes 4 moves for each player to see how many branches follow. It will then weigh each next move in the list based on that. If you want to turn this off completely, just select equal, which means it will not look at following moves at all. 

Variation arrows from transpositions

category: Analysis/Study - Advanced
values: yes / no - defaults to yes

This feature is showing variation arrows for transpositions, as defined in the Transpositions behavior feature.

Next move behavior for variations

category: Analysis/Study - Advanced
values: Normal / Hybrid / Force choice- defaults to Normal

This feature will change how the next move arrow works when there are multiple variations to choose from:

  • Normal - normal behavior
  • Hybrid - this will highlight the variation list, asking you to make a choice. Trying again will just pick the selected choice in the list. On desktop you can cycle between choices with the Shift key or just click on the move you want. On mobile you can tap the choice you want.
  • Force choice - this will show a popup with the list of variations and transposition moves. (similar to the Chessbase behavior)
    • On desktop you can:
      • select the one you want with up/down arrows.
      • to execute a choice: Right-arrow key, Enter, the OK button or right-click or double-click or click on an already selected option
      • to close the popup: Left-arrow key, Esc, the close button or clicking outside the popup 
    • On mobile you can:
      • select the one you want with a tap
      • to execute a choice: tap a selected variation
      • to close the popup: tap the close button or outside the popup

Notes:

  • the variation list might be affected by the Transpositions behavior feature. You could get a list of transposition variations, but then the Shift key will not work to shuffle between them. This is (so far) by design, because transposition moves are not actually moves, they are jumps to other branches.
  • if you have enabled the Lichess game behavior Scroll on the board to replay moves, then that will ignore this feature, per design. So if you want to just quickly go up and down your move tree without having to enable/disable this feature, use the mousewheel or the equivalent touchpad moves.
    • note that this was enabled before with a separate feature that mirrored the native Lichess functionality. That tool has been removed in v2.3.136.

PGN Editor

category: Analysis
values: yes / no - defaults to yes

This feature adds a new entry in the Tools menu called PGN Editor. Clicking on it will open a text editor where PGNs can be pasted, manipulated and copied from. The PGN Editor is actually a popup over the page you are on. For technical reasons, for the Evaluate function to work you must be in an analysis/study page, therefore if you open PGN Editor in any other page it will jump to Analysis Board page and show the popup there.

Functionalities (short version):

  • Merge - combine all games into one (or few) PGNs
  • Normalize - transfer all moves after transpositions to the first instance of the position
  • Denormalize - clone all variations from transposing positions into all instances of those positions
  • Split - split all variations into their own one line games
  • Search - find games with particular characteristics
  • Result - remove all games that were not marked as found by Search
  • Cut - remove things
  • Evaluate - add evaluation comments to the final moves in every variation
  • Extract - extract custom information from existing PGN as a downloaded file
  • Count - display the number of games and how many moves in the PGN
  • Cancel - cancel currently running operation
  • Copy/Undo/Redo/Clear/Upload/Download - operations with the content of the PGN Editor

Functionalities (oh, boy!):

  • Merge - this button will take multiple games in PGN format and merge them into as little games as possible. For example, all games starting from the same position can be merged into one. Also, games that start from a position present in another game will be merged there. Games starting from positions not present in the others cannot be merged.
    Example: 
    Original PGN content:
1.e4 e5

1.e4 d5

After merge:
1. e4 e5 ( 1... d5 ) *
​
  • Normalize - this button will, for each game in the PGN, take all moves made from a particular board position in any variation and group them in the first occurrence of that position.
    Example: 
    1. e4 (1. d4 e5 2. e4 a5) 1... e5 2. d4 h5 

(After both paths e4 d5 d4 and d4 d5 e4 the same position is reached (a transposition). In the original PGN the first path move from the common position is 2...h5 and in the second path 2...a5.)

Normalizing this will result in:

 1. e4 ( 1. d4 e5 2. e4 ) 1... e5 2. d4 h5 ( 2... a5 ) 

(both a5 and h5 moves from the mainline position, while the d4 path will stop at the common position, with no following moves.)
  • Denormalize - this button will, for each game in the PGN, take all the moves made from a particular board position and add them as actual moves in the PGN. The purpose is to create all possible lines as actual moves.
    • This is the reverse of the Normalize function, but Denormalizing and Normalizing doesn't yield the original PGN.
    • Denormalization will increase the size of your PGN.
  • Split - this button will, for each game in the PGN, split each variation into their own games. Example: 1. e4 e5 ( 1... d5 ) will be split into 1. e4 e5 and 1. e4 d5
    • This is the reverse of the Merge functionality above.
  • Search - this button will prompt for a string, then search the games for it.
    • The string can be part of a FEN or a PGN (wildcards * for anything and ? for one character are supported)
      • when searching for a PGN, the original PGN is searched as well as with comments and annotations removed and then with move numbers removed, see Examples
    • Another option is to search for PGN tags, like this: Black=totalnoob69 or opening*=Gass.
      • *= means contains, the tag name is case insensitive
    • You can search by index like this: Index=3 (the third game in the list)
    • You can also search the invalid games like this: Invalid (this marks invalid games as found)
    • Yet another option is to search for ply (half-moves), like this: ply=10, ply<30, ply>15
    • Searching is possible on eval comments generated by Evaluate, like this: eval=1, eval>0.4, eval<-1.5
    • The games that match will have a new PGN tag added (called Found) in the text area and also will be copied to the clipboard. Searching will remove previous search Found PGN tags.
    • Do not use full FEN strings, only the first four elements (board, who's turn it is to play, castling and en passant square) are supported (no clock values)
    •  Examples:
      • searching for rnbqkbnr/pp2pppp/2p5/3p4/2PP4/5N2/PP2PPPP/RNBQKB1R b KQkq - 1 3 will fail (because it has half clock and ply - 1 3 - at the end)
      • searching for rnbqkbnr/pp2pppp/2p5/3p4/2PP4/5N2/PP2PPPP/RNBQKB1R b KQkq would work for the games that reach that position
      • same for:
        • rnbqkbnr/pp2pppp/2p5/3p4/2PP4/5N2/PP2PPPP/RNBQKB1R - FEN search
        • pp2pppp - FEN search
        • rnbqkbnr/*/2p5 - FEN search
        • 5?2 - FEN search
        • ECO=C40 - PGN tag search
        • c4 c6 - PGN search
        • ply=10 - search games with exactly 10 ply (5 moves for each side)
        • ply>30 - search games with more ply than 30
        • ply<20 - search games with less play than 20
        • eval>1.5 - search games where a final move has been evaluated to more than 1.5 eval
          • note that if you have several branches in the games, it will show as found if at least on branch terminates in a matching eval.
      • for a game that looks like this: 1. e4 {some comment} ...c5 2. c3 , all of the following PGN search patterns will find it:
        • 1. e4 c5 2. c3
        • e4 c5 c3
        • e4 * c3
  • Result - this button will remove all games that are not marked with a Found tag and remove the Found tag from those found. The flow is: you press Search, it finds some games and marks them with a Found tag, you press Result and only the found games remain in the text area.
    • Warning! If you have not searched for anything before (or if you press the button twice), it will clear the text area
  • Cut - this button will remove things based on the prompt you give it:
    • if it contains "tags" it will remove all tags from games
    • if it contains "comments" it will remove all comments from games
    • if it contains "annotations" it will remove all annotations from games
    • if it contains "result" it will remove all found games (the opposite of the Result functionality)
    • if it contains "ply <some value>" it will remove everything after the specified number of ply (half moves), on every branch, in every game
    • if it contains "eval<operator><some value>" it will remove branches that have that particular eval, on every branch, in every game (i.e. eval<3 will cut everything with evaluation less than 3)
      • operator can be < > or =
      • this works with the comments for evaluation from the Evaluate command! It does not evaluate the position itself. 
    • if it contains "junk" it will attempt to eliminate from the text what appears to not be a valid PGN game. Use it after copy pasting text from chess webpages when you want to clean up the result.
    • the above patterns can be combined
    • Example: tags, comments, annotations, ply 10
  • Evaluate - this button will evaluate the end positions of each game in the PGN, adding an eval: <value> comment
  • Extract - this button will extract various types of information based on your prompt
    • if it contains "fen" it will extract the FEN for each move in each game. Games will be defined by their index, followed by the list of unique FEN positions
  • Count - this button will count the number of games and total number of moves in them
    • This is done automatically after most operations.
  • Cancel - this button will cancel the current operation (for example when trying to merge tens of thousands of games, it may take a while)
  • Copy - this button copies the content of the text area in the clipboard
  • Upload - this button will ask for a file to upload to the text area (dragging files and dropping them directly to the text area works as well)
  • Download - this button will download the content of the text area as a pgnEditor_<currentTime>.pgn file
  • Undo - this button will undo the changes in text area (equivalent to pressing Ctrl-z)
  • Redo - this button will redo the changes in text area (equivalent to pressing Ctrl-y)
  • Clear - this button will clear the text area as well as the entire undo/redo history.

Notes:

  • Games with no moves and no tags will be removed. Games with moves and no tags (example: after Cut "tags") will have an Event tag with the value "exported by LiChess Tools". This is there so that various import software, including Lichess studies, process the list correctly, not an unnecessary blurb.
  • Normalize changes the flow of the moves. Don't expect the normalized PGN to have the same structure as the original. This functionality is particularly for grouping the moves from a particular position in one place.
  • Tag search will search the exact string in the tag, but will ignore whitespaces. Use the *= syntax to search for containing strings (i.e. opening*=Gass)
  • I am thinking of a meta-search option, where you can search for pins, sacrifices, pawn structures, etc., but I don't know how many people use that. Let me know if you are interested.

Enhanced PGN import

category: Analysis
values: yes / no - defaults to yes

Another important feature of LiChess Tools, this one makes changes to the import mechanism in the Analysis board page. You can now import multiple PGNs at the same time and they will get merged into a single one. It is great for creating repertoires, analyzing the games of a specific player or creating Extended Interactive Lessons. From the Analysis board you can go to options and turn it into a study. Note that the recommended way to perform PGN operations is now the PGN Editor.

As part of this you will also get the Escape key functionality for the FEN and PGN inputs, clearing the focus and allowing key shortcuts again.

Notes:

  • the import mechanism in Analysis board ignores comments from the PGNs. This is a lichess limitation and to get around it I would have to rewrite the entire thing. Use PGN Editor for work with commented PGNs.
  • the import in Analysis board is different from the one in Studies. The one in Analysis board is client based, while the one in Studies is on the server. That is why I can only alter the one in Analysis board.
  • there is no logical limit to how many PGNs to import, but the Analysis board has a limit of about 5000 moves. Also, the API to turn it into a study has a size limitation, so if you try to turn a huge analysis to a study and it fails, know that it is too big. Highlighting moves on huge PGNs also is slow. So use with care.
  • From v2.1.21 the merge mechanism will check if the PGNs start from different positions and only merge what starts from the same position. Previously the merge seemed to be working, but it made no sense from the standpoint of PGNs, resulting in an invalid PGN that cannot be loaded.

Friends box

category: Friends
values: Default / Menu / Open / Hidden - defaults to Menu
Needs log in

This has evolved a lot. Originally it was automatically opening the friends box, but then several requests came that complained about the placement of the friends box and how distracting it was. So the current version of the feature can either:

  • Default - the default behavior of the friends list: stays in the bottom-right corner, only updates when opened, which requires a user click.
  • Button- the friends box is removed and replaced with a friends button, similar to the Notifications one. It will update in real time with the number of friends online and will change color when any one of them is playing.
  • Menu - the friends box is removed and replaced with a friends menu. It will update in real time with the number of friends online and will change color when any one of them is playing. It will also show the number of playing friends in a small blue square.
  • Open - the default behavior of the friends list, but it will open automatically when loading the page
  • Hidden - the friends list will disappear completely

Notes:

  • clicking on the main menu item directly will open the friends page, but not on mobiles
  • when the width of the page goes under 972px, menus are grouped under a hamburger menu. The Friends menu will disappear and become a button in that case.
  • playing friends will have the time control icon on the left of their name in the friends menu.

Live friends page

category: Friends
values: yes / no - defaults to yes
Needs log in

The default lichess behavior is to load whatever list of friends you have and then leave it at that. With this LiChess Tools feature the friends list will be updated in real time, while also allowing filtering by who is online, who is playing, who is active (played in the last year) and (only if player voice alerts are enabled) which player is muted. 

Each player will have a TV icon as well and (only if player voice alerts are enabled) a mute/unmute button to include them in the alerts. (see Sound and voice alert with friends start playing)

Sound and voice alert when friends start playing

category: Friends
values: Ultrabullet / Bullet / Blitz / Rapid / Classical / Standard only- defaults to none
Needs log in

This feature will beep and spell out the player name and type of game they started. The type of the game can be chosen from the values. The definition of the game type comes from lichess, which assumes a typical game will be around 40 moves.

You can also choose which specific players to have the alert for, by going to the friends page and clicking the mute buttons. (see Live friends page)

If you select Standard only, then the alerts will only be sounded for the Standard variant of the game.

Notes:

  • Chromium (and other) browsers disallow sounds from page load until the user does something like click on the page. In case an alert is attempted and audio is not allowed a red icon will appear next to the top buttons, which will disappear once the audio is allowed again.
  • The audio icon is part of lichess, which will also display it once sound is attempted but not allowed. At this time, the lichess icon will NOT disappear unless you click on it, different from how LiChess Tools uses it for alerts.
  • If you are playing a game on another tab, the sounds will not be played if the Quiet mode on all tabs feature is enabled.

Practice against moves from Opening Explorer

category: Analysis/Study
values: yes / no - defaults to yes

This is another great feature that allows you to play against whatever is selected in the Opening Explorer. Select masters to play against the best players, select Lichess and select the range of player to train against and select specific users to test your strategies against their played games. The feature can be turned on/off not only from Preferences, but also from the Explorer config screen.

In order to use this feature you need to go to the Analysis board or a study, open the Explorer and click on the icon next to the Masters tab, the one that looks like an arrow hitting a target.

One extra feature of this tool is making a random Explorer move if clicking on the total row on the bottom. It effectively makes a move as if Explorer Practice is enabled and it's the computer to move.

The feature is unavailable in the Analysis screen of a running correspondence game.

Notes:

  • you can toggle Explorer Practice by clicking on the arrow in the target button, but also with shift-L, to make it easier to switch between computer evaluation and the practice mode
    • the keyboard help popup shows this shortcut if the feature is enabled
  • by default the moves in the explorer window will get hidden in Analysis board, but not in studies. The motivation being that one is more like a game, with no lasting effects, the other is research. See Explorer Practice options in order to change that.
  • if the Show emojis when out of moves feature is enabled in Explorer Practice options, then at the end of a run, when there are no more moves available in the Explorer for the feature to make, a quick computer analysis will be run to show you a smiley (happy or sad).
  • When Explorer Practice is running, the normal Practice button will be hidden, in order to avoid both confusion and pressing on it accidentally on mobiles when wanted to go back to start.

Explorer Practice options

Options for the Explorer Practice feature:

  • Show emojis when out of moves - set by default - will run a short computer analysis when out of moves in Explorer Practice, then show an happy/sad emoji based on the eval.
  • Click Explorer Σ to make a move - set by default - clicking on the Total row in Explorer will make a random move, weighted by the probabilities of the move (basically a one turn Practice move)
  • Show opponent name - disabled by default - this will show a name next to the move made by the computer, giving you a hint about who played the same move in the same position
  • Explorer data in Study - set by default - this will show explorer rows while doing Explorer Practice in a Study page
  • Explorer data in Analysis - disabled by default - this will show explorer rows while doing Explorer Practice in the Analysis board page

Highlight moves in analysis

category: Analysis/Study - Advanced
values: Last move in each variation / Not commented last moves / Transpositions to current move / Highlight board when out of main line / Highlight pieces when out of main line / Highlight variation depth - defaults to Last move in each variation, Not commented last moves, Transpositions to current move

This feature will highlight moves in the analysis move list. Each option highlights a specific thing:

  • Last move in each variation - for each branch, highlights the last move
  • Not commented last moves - for each branch, highlights the last move that has no comment and is not a mate
  • Transpositions to current move - highlights positions that are identical to the current one in the same PGN (see Behavior of transpositions to change what the extension considers a transposition)
  • Highlight board when out of main line - dims the board a little when moving out of main line (similar to chess.com)
  • Highlight pieces when out of main line - dims pieces that moved from the mainline position (similar to Chessmaster 10th edition)
  • Highlight variation depth - it will change the display of the moves so that they have different colors depending on the variation depth level.
    • this replaced the Move colorizer theme that existed until version 2.3.
    • Warning: this may have adverse performance effects on really large PGNs (1000+ moves)
  • Highlight checks to kings - highlights moves that place a king in check

Extra context menu options

category: Analysis/Study
values: Copy branch as PGN / Engine evaluation for last moves / Highlight all transpositions / Remove superfluous / Show context menu when no moves - defaults to Copy branch as PGN, Engine evaluation for last moves, Remove superfluous, Show context menu when no moves

This feature will add extra options to the Analysis board/Study move list context menu. The options are:

  • Copy branch as PGN - this works in Analysis board and Studies and turns all the previous moves of the line you are in, plus all the branches after the move, into a PGN in your clipboard.
    • If you press the Shift key when clicking on Copy branch as PGN the copied PGN will start from current position, not from the beginning of the original chapter.
    • If you press the Control key when clicking on Copy branch as PGN each variation branch will be copied as a separate PGN game.
    • If you press the Alt key when clicking on Copy branch as PGN then the PGN will get copied only to the current position.
    • You can use it to split large PGNs by copying variations into their own chapters, then maybe deleting them from the original one.
  • Engine evaluation for last moves - this works only for Studies, because it involves move comments. It will go through all of the uncommented last moves and create a comment in the form eval:<evaluation> using the local computer engine. The evaluation will stop at the engine depth selected in Preferences (see Custom analysis engine depth).
  • Highlight all transpositions - this works for both Analysis board and Studies and highlights all moves that have transpositions. You can configure what a transposition is in Preferences (see Behavior of transpositions)
  • Remove superfluous - for purposes of saving space, this removes the following entries from the context menu:
    • Annotate with glyphs - there is a button for this under the board
    • Comment on this move - there is a button for this under the board
    • Copy variation PGN - LiChess Tools already provides a better version
  • Show context menu when no moves - this allows for showing the context menu when there are no moves in the tree. When the board starts from a FEN position, there is a little "..." element that you can right-click. When the board is completely empty, there is a row that highlights when you hover and you can right-click for the context menu.

Move assistant

category: Analysis/Study
values: yes / no - defaults to yes

This feature will add a little eye icon button () in the computer evaluation window, next to the settings. Clicking this button will enable the Move assistant, which will show, when selecting a piece, the evaluation on each possible destination square.

The colors represent the quality of the move. Green is good, Red is bad, Yellow is not recommended.

  • The small circle inside the destination square represents the evaluation relative to the worst and best possible move.
    • This means that if the best move achieves equality and the worst move loses 0.2 pawns, the first one will appear green and the other red.
    • This is the less important metric, so it's not that serious that you can't see it when the destination square is a capture. 
  • The border color of the destination square represents the inaccuracy/mistake/blunder metric.
    • While this is still a gradient, anything that will cause a 3 pawn loss from the best move will appear as red, 2 pawns will get you orange, 1 pawn will get you yellow and only under you will get green.
  • The bar inside the border represents the WDL (win-draw-loss) chances after making the move. Again, red is loss, green is win and yellow is draw.
    • Take these with a grain of salt, as the WDL values of Stockfish are not that relevant.

This is not meant to replace, but be complementary to the computer analysis. Move assistant enabled value is persistent on page reloads.

Once a piece is selected, the background engine will run using the configured thread count and hash - for 90 seconds and then stop. During that time the color of the squares will be updated as the evaluation changes. 90 seconds corresponds to a depth of about 20 on my older laptop in a middlegame position. Unselecting pieces will stop the processing.

Obviously, this consumes processor power, so if you are on a mobile device or a laptop it will consume battery. Running both local computer analysis and Move assistant will use CPU for both processes. Opening multiple analysis browser tabs will use resources for each tab in which a piece is selected.

Sometimes destination squares will not be colorized. For example, this is the case for mate threats, where moves that do not interfere with the mate will not be evaluated. 

Custom analysis engine depth

category: Analysis/Study - Advanced
values: a number representing the depth - defaults to empty

This feature will allow you to set a depth level for the computer engine. If the current move has a smaller depth level (let's say it was cached in the cloud with that level) it will start the local computer evaluation. Once it reaches the set depth, the engine will stop. You can restart the analysis by pressing the little + button next to the computer engine depth indication.

This feature is also used as the level at which to consider evaluation done for the Engine evaluation for last moves option (see Extra context menu options - Engine evaluation for last moves and PGN Editor - Evaluate)

This feature is also used in Practice, where it controls the level of play (See Custom analysis engine options)

Custom analysis engine options

category: Analysis/Study - Advanced
values: Ignore cloud data for evaluation / Ignore cloud data for external engines / Apply in Practice mode / Fix external engine 503 errors - defaults to none

This feature controls the behavior of the analysis engine used in Analysis board and Studies.

Options:

  • Ignore cloud data for evaluation - Use this option to never show computer evaluations from the cloud. For example if you want to not see evaluations over a certain level or in practice mode. You can also toggle this on/off from the Analysis board hamburger menu.
  • Ignore cloud data for external engines - just like the one above, but only applies when you have configured an external engine. This makes sense, as the evaluation in the cloud is computed by Stockfish and you might have another engine altogether. The Analysis board hamburger menu will only show this feature when the current active engine is external.
  • Apply in Practice mode - Use this to apply both the custom engine depth and the no cloud option before in Practice mode. This means you can play against a computer of your own level and ignore cloud and table bases. You can also toggle this on/off from the Analysis board hamburger menu.
  • Fix external engine 503 errors - When you configure external engines, you sometimes get 503 errors, meaning that the API was not reachable. This might be because you haven't started your Lichess Local Engine utility, but more often it's just a timeout. This will disable the alert and just silently retry after 5 seconds.
    • You probably will never need this, as the reason it was created was to fix a bug that had a different solution. But it might still be helpful if you have unreasonable timeouts.

LiChess Tools analysis engine threads

category: Analysis/Study - Advanced
values: a number representing the number of threads - defaults to 1

LiChess Tools uses a local analysis Stockfish engine for various features, like Move Assistant or PGN Editor. This configures how many threads to use for that engine. Note that this value applies per open tab, so if you have Move Assistant enabled in Analysis board, for example, and select one piece in multiple tabs, the number of threads will multiply with the tab count.

LiChess Tools analysis engine hash

category: Analysis/Study - Advanced
values: a number representing the number of megabytes in the hash - defaults to 128

LiChess Tools uses a local analysis Stockfish engine for various features, like Move Assistant or PGN Editor. This configures how many MB or memory to use for that engine. Note that this value applies per open tab, so if you have Move Assistant enabled in Analysis board, for example, and select one piece in multiple tabs, the size of the hash will multiply with the tab count. 

Show next moves from transpositions

category: Analysis/Study - Advanced
values: yes / no - defaults to yes

When you get to a certain position in the PGN which has transpositions, the next moves from those transpositions will be displayed under the move list, just like for a fork into multiple variations. You can configure what a transposition is, as well as if you want multiple identical moves to be shown or not and if the moves in this list should be considered by Ctrl-right and Extended Interactive Lessons in Preferences (see Behavior of transpositions)

Extra key shortcuts

category: General - Advanced
values: yes / no - defaults to yes

This feature adds extra key shortcuts:

  • to the Analysis board/Studies: 
    • first it changes the functionality of the i, m and b keys to work in Analysis board as well as Studies and to use all variations, not just the main line as the original lichess behavior. The functionality of these keys is to jump to the next inaccuracy, mistake or blunder, respectively
    • it adds the same functionality to Alt-i, Alt-m and Alt-b, only for the opponent
    • g and Alt-g cycle through good/brilliant/interesting moves
    • if Explorer is shown:
      • shift-T will cycle between Masters and Lichess database
    • it adds the ability to select any of the moves in the available moves lists:
      • press . (dot) and then a digit and it will make to the nth next move available in the PGN (see Transpositions behavior if you want to select moves from transpositions, too)
      • press Ctrl-. and then a digit and it will make to the nth next move available in the computer evaluation list
      • press Shift-. and then a digit and it will make to the nth next move available in the opening explorer list
    • ` (the key before 1) and then f will freeze/unfreeze the board - this means you can still move pieces, but the board will appear as when it was frozen. Helps with visualization.
    • ` (the key before 1) and then r will activate the random chapter navigation button. You need to have that button enabled in order for it to work.
    • in correspondence games, in analysis mode, the Backspace key can be used to return to the current position
  • in the Board Editor screen:
    • 1-8 to select the buttons under the board (select, pawn, knight, bishop, rook, queen, king, erase)
    • Shift+1-8 to select the buttons above the board (same thing, different color)
    • c to clear the board
    • p for the starting position
  • everywhere:
    • ` (the key before 1) and then h will toggle the header of the site. It's a pretty niche feature, but it helps if you are distracted by the header, like when you're playing in 3D mode and the header overlaps the taller pieces.

Notes:

  • the keyboard help popup (Shift-?)  shows all of these shortcuts if the feature is enabled

Additional glyphs

category: Study - Advanced
values: yes / no - defaults to yes

This feature displays an extra # glyph on mate moves. Also it enlarges glyphs on mobile devices.

Notes:

  • The lichess UI allows for multiple glyphs on the same move. The one shown on board will just be the first one in the list.
  • Originally this feature was created to show annotations that lichess was not showing. From September 2023 lichess started natively showing all annotations, therefore eliminating the need for much of the functionality of the feature.

Extra analysis charting

category: Analysis/Study - Advanced
values: Material / Principled / Max tension / Max potential / Find interesting moves / ... more moves / Local eval / Accuracy / Sharpness / Chart smoothing / on Eval gauge  - defaults to Material, Principled, Max tension, Find interesting moves, Accuracy, Chart smoothing, on Eval gauge

When not logged in, Local eval, Find interesting Moves and ... move moves are enabled by default.

This feature is adding extra lines on the computer analysis chart. In the search for good/great/brilliant moves - as shown on chess.com and requested by a lot of players, the lines drawn over the normal computer eval chart are meant to give some indication of what is going on without asserting an exact significance of the values. The lines are purposefully left without explanation, in the hope that players recognize some aspects of the game in the numerical values charted and their intersection.

For example, many of the moves generally considered brilliant see a steep increase in evaluation, but a steep decrease in material and/or principled position, indicating a sacrifice or an unnatural seeming move that wins the game.

If Find interesting moves is enabled then an extra entry in the analysis summary shows the total of interesting/good/brilliant moves in the PGN. With ... move moves the extension will endeavor to show which moves are good, best and brilliant, but take it with a grain of salt.

Options:

  • Material - green dashed line shows a more classical material difference between the sides
  • Principled - blue dashed line shows how principled the position is. This means stuff like square control, freedom of movement, developed centralized pieces, etc, but excepting material value.
  • Max tension - red dotted vertical line shows the point of maximum tension in the game. Tension here is defined as the total material that can be captured by both sides in just one move.
  • Max potential - green dotted vertical line shows the point of maximum potential (similar to tension, but it superficially looks at capture chains)
  • Find interesting moves - if enabled, this will show good/brilliant/interesting moves in the analysis summary as a link that cycles through them. It will also automatically find some categories of interesting moves and mark them with !? glyphs (if not already annotated). The glyphs added by this algorithm will be temporary (not persisted on the server).
  • ... more moves - this will automatically calculate good/best/brilliant moves. It will also change the behavior of the good/brilliant/interesting moves link to only search brilliant and interesting moves. The G shortcut will work as before.
    • this uses both the server evaluation and the local computer eval. If you run the computer engine locally it will change the glyphs in real time.
    • every move that isn't bad is good, so you will get A LOT of good moves. 
    • as I don't know which is the best move, only the eval associated with it, a "best move" is a move as good as the previous evaluation. This means more moves could be "best" and depending on Fishnet, you will sometimes need to run local eval to get a "best" or to clear a false one.
  • Local eval - if this is set, the local evaluation will also generate a yellow line chart, with a semitransparent background under it. This will also add charting to the Analysis board, once the local engine has been started, and it also updates various other chart lines with local eval, if the depth is higher than the server one.
    • the yellow chart shows the evaluation of the current variation branch, not the mainline like all the other chart lines. Meaning that the chart line will appear and disappear depending on if the current branch has computer evaluations.
    • if the chart is local only (no server eval available or run yet) this will show inaccuracies, mistakes and blunders as well. The nice summary on the right side that you get with the server eval will NOT be there.
  • Accuracy - if this is set, the accuracy of the moves of the side the board is oriented from is charted with a magenta color. More on the actual formula here: Lichess Accuracy Metric
  • Sharpness - if this is set, the sharpness of the position is charted with pink.
    • note that this chart depends on Explorer values, so it will only show as far as there are records in the Lichess Explorer database and you have visited those positions with Explorer enabled.
  • Chart smoothing - if this is enabled, the chart will be smoothed, removing sharp spikes. 
  • on Eval gauge - if this is set, two lines (one green and one blue) will appear on the evaluation gauge when it is visible, changing with the values of material and principled values, respectively.

Notes:

  • I called it "interesting moves" because there is no objective good/brilliant move algorithm. If you want those, enable ... more moves.
  • If you are not logged in, Local eval, Find interesting moves and ... more moves will be enabled by default.

Study links options

category: Study - Advanced
values: Video popup - defaults to Video popup

This feature handles links inside study comments. It enables Chessable/Chessmood kind of courses, where you can play the course or do the puzzles or research lines while the video is running on top.

Options:

  • Video popup - if the link is to a recognized video provider, clicking on the link will open a popup where the video runs. The popup can be resized, moved and the settings for this will be persisted for the current device.
    • only YouTube, Vimeo and Twitch videos are supported for now, because Lichess blocks anything else
    • start timestamps are supported for all three types of videos
    • for YouTube links an "e" parameter just like "t" can be use to end the video at that timestamp. Example: https://youtu.be/<someId>?t=300&e=400
    • this does not work in Firefox because they don't support credentialless iframes
  • Open links to studies in same window - this makes links from move list comments which lead to studies to open in the same window. The reason for this is that you want to continue a variation that is split into multiple chapters or that leads to another study in the same window, especially in Extended Interactive Lesson mode. Now you can utilize this with the Bookmarks option of the Move List Options feature to jump from move to move and chapter to chapter.

Notes:

  • the video popup will snap to the margins of the screen, so if you place it in the bottom right corner and resize the window, it will keep a fixed position relative to the bottom and the right sides of the screen
  • if less than 50% of the video header is outside the screen, the popup will reset to original position: center of the screen with 640x480 size
  • the video link can have a start time parameter which should be respected by the video popup

Learn from your mistakes in Studies

category: Study - Advanced
values: yes / no - defaults to no
Needs log in

This feature will add a Learn from your mistakes button for study chapters that have a server computer analysis, just like for game analysis.

Change chapter names from PGN tags

category: Study - Advanced
values: yes / no - defaults to yes
Needs log in

This feature will give you the option to quickly change the name of a study chapter to either the content of its Event PGN tag or the White and Black PGN tags combined when editing the chapter. Sometimes people want to import a series of PGNs into a study, but with the default lichess behavior only the first chapter gets a proper naming from the PGN tags, the rest getting the default Chapter 2, Chapter 3... etc name. With LiChess Tools you can go to each of them, edit them by clicking the cog button next to their names, then choosing with one click the name the chapter should have.

A newer functionality of this feature is to suggest a name based on the mainline moves. It should help to structure studies with a variation per chapter.

Study chapter navigation controls

category: Study - Advanced
values: yes / no - defaults to yes

This feature will add a set of buttons at the end of the chapter list in a study. The buttons navigate to the first chapter, previous chapter, random chapter, next chapter and last chapter respectively.

Notes

  • You can add in the title a rnd:N text, where N is a percentage number between 0 and 100. This will change the chance that a chapter will be selected by the random button.
  • If the mechanism above is used, the current chapter can be chosen (so basically remains the same) by pressing the random button

Sticky study Preview mode

category: Study, Interactive Lesson chapters - Advanced
values: yes / no - defaults to yes

This feature will keep you in Preview mode as you move from one Interactive lesson chapter to another. As one trains using interactive lessons, at the end of one they are presented with the option to move to the next chapter. The default lichess behavior is to show you the chapter in edit mode (assuming you are the owner or a contributor) which requires you to press the Preview button all the time to continue training, after also having seen the move list. Now you can press Preview once and play chapter after chapter with no hassle.

This is preserved also between page reloads, so you will have to manually unset Preview mode to edit a study Interactive Lesson.

No spoilers

category: Broadcasts - Advanced
values: Broadcast - defaults to none

This feature will hide the scores in the games in broadcasts. It will also add a "No spoilers" checkbox in broadcasts so you can set/unset it from there.

Show game opening names

category: General
values: For large board / For minigames / In Explorer - defaults to all

This feature will load the lichess opening name for games and display it. When watching a game or analyzing it, the opening will be shown under the player names. For mini-games (the tiny ones that appear when hovering over a game link or put somewhere on the page) it will appear at the bottom. For Analysis board, if the wiki side is empty, it will show the opening there, assuming the position is in the local cache. For Explorer, if open and the option is set, the extension is going to attempt to add just the relevant information to the existing opening name.

Options:

  • For large board - this will enable showing the opening name for Analysis Board, Studies, Board Editor, TV games.
  • For minigames - this will enable showing the opening name for games shown when hovering over a player name, games shown in the Current Games or broadcasts and any other games shown in small board format.
  • In Explorer - this will enable showing extra opening information in the Explorer header, next to the normal opening name coming from Lichess, if applicable.

In the very rare cases where the same exact position is found with colors reversed, the name of the opening will appear with a little (R) at the end.

This should work for Analysis Board, Game analysis, Studies, TV games, Board Editor, Broadcasts, Swiss tournaments, etc.

Show pawn structures

category: General
values: Enabled / Only named structures / Fuzzy search - defaults to Only named structures, Fuzzy search 

This feature, similar to Show game opening names, shows the pawn structure of the current game position. Because it's all very local, with no API requirements, it also works for current game pages.

An entire blog post about it can be found here: The Pawn Structure feature in LiChess Tools

Options:

  • Enabled - enables or disables the feature - disabled by default.
  • Only named structures - show only structures with names. This is enabled by default, but of course will not work unless enabling the whole feature. If you disable this, you will see a very technical name for each and every move.
  • Fuzzy search - this will show named pawn structures within a certain proximity. It helps to determine the general structure, even if it might not be totally accurate.

Notes:

  • This feature has a lot of potential, as the pawn structure is something that informs medium to long term plans. Knowing these will help you grow a lot as a chess player.
  • If the pawn structure is named, it will show as a link which can be clicked to learn more.
  • If the structure is not found in the available list, it will be searched from the opponent's perspective, in which case it will be shown with a (R) at the end if found

Show player country flags

category: General
values: yes / no - defaults to yes

A very early feature of the extension, it has also evolved a lot. The latest version is much more efficient in terms of how it gets the data from the server and adds flags to almost every place where a user link is shown.

In order to not use too many resources, the flags for players are cached for 10 days. If you want to clear the cache, go to Preferences and turn it off and on again.

Show player score deviation

category: General - Advanced
values: yes / no - defaults to no

This feature will add the rating deviation to some of the ratings in the user popups (the one popping up when you mouse over player links). The deviation will be either red, normal or green based on the progression in the latest games. Red means loss of rating, green means increase in rating.

Chat/forum options

category: General
values: Paste image support / Large one emoji message - defaults to all
Needs log in

This feature applies to the inbox chat and writing forum posts. There is another type of chat (below) used in games, studies and team pages.

Options:

  • Paste image support - allows for pasting images directly in the chat or the forum. Here, by "directly", I mean that it uploads the image to Imgur, retrieves a URL for it and pastes the resulting link. From there, Lichess has support to display the image.
  • Large one emoji per line - in Inbox chat, when you have only one emoji message, like 👍, it will appear larger

Team/Study chat options

category: General
values: Highlight URLs / No length limit / Image support / Team chat notifications - defaults to all
Needs log in

This feature adds some much needed functionality to the chat used in games, studies and team pages:

  • Highlight URLs will turn URLs in the chat text into links
  • Image support will show images instead of text in the highlighted URLs (assuming the URL is of an image) - requires Highlight URLs
  • No length limit will allow you to write as much as you want and then split the message in 140 characters slices - because that's what the lichess server allows
  • Team chat notifications will show you notifications when messages are added to the chat of a configured team. The teams list will now show a notification bell next to each team that will toggle notifications for that team. A maximum of three teams can be configured for notifications. By default, no team is configured for notifications.
    • If you stay on the team list page long enough, the notification bell button for enabled teams might show a number on it. That's the number of people active on the chat (which includes people who have enabled notifications for that team) if there are more than 1. Obviously, in order for this to work, you must be one of them.

Notes:

  • this is different than the team "Subscribe to team messages" toggle, which is a native lichess control for receiving mass team messages from the team admins.
  • visiting the page of a team will reset the notification for new messages (will mark all as read).

Better exported board image

category: Analysis/Study/Board editor - Advanced
values: yes / no - defaults to yes

This feature will override:

  • the default share Board button in studies
  • Screenshot current position in game analysis
  • SCREENSHOT button in the board editor

It will also add a SCREENSHOT button to the analysis board with the same functionality.

When clicked, the button will export not only the board position, but also the arrows and anything else drawn on the board in the current position. You retain the original behavior by right-clicking the button and opening in a new tab or window.

If you Shift-click, the board will be rendered without coordinates.

Link to current analysis

category: Analysis - Advanced
values: yes / no - defaults to yes

You ever wanted to send to a friend the state of the Analysis Board you've been working on for? Now you can. Under the PGN label of the text under the Analysis Board you get a small link icon that contains the link to the Analysis board with the same PGN, orientation and move number.

Notes:

  • the maximum allowed size for a browser URL is 2048 characters. If that size is exceeded, the link will not be available.

Autosave analysis for reload

category: Analysis - Advanced
values: yes / no - defaults to yes

This happened to me a lot of times: I am analyzing something and I accidentally swipe the touchpad or press some key that reloads the page. All my work is gone! This feature keeps track of the latest modifications of Analysis board PGN and reloads it on page load. It is your choice if you want to import it or not by going to the PGN text area and clicking the Import PGN button.

Notes:

  • The saved PGN as well as the Analysis board import text area content will be overwritten by whatever move you make, so if you fail to import the PGN and make a move, you lose the data. 

Opening explorer player features

category: Analysis/Study - Advanced
values: Me button to switch to your player - defaults to Me button to switch to your player
Needs log in

This feature controls the choice of the player in the Opening Explorer.

  • Me button to switch to your player - adds a tab next to Masters database, Lichess and Player named Me. If you click on it the choice of the player will toggle from your user to the last selected user.  The button can be shown/hidden from the Explorer config screen as well. 

Notes:

  • The Me button will appear only if you have any another players configured for the Player tab name list

Toggle snapshots of Explorer settings

category: Analysis/Study - Advanced
values: yes / no - defaults to yes

This feature will add a new button called Snap! in the Explorer settings of the Lichess database tab. This will allow to save the current Explorer settings under a name, then quickly toggle between the various sets saved by clicking on the same tab, which will now change text to reflect the name of the selected "snap". You can easily delete existing snaps and create others.

Resize Explorer

category: Analysis/Study - Advanced
values: yes / no - defaults to yes

This feature will allow you to resize the Explorer window height inside the analysis tools, therefore controlling how much space it takes when opened. Just drag the divider above or below the Explorer to change the size.

Notes:

  • The mechanism works very differently from desktop to mobile (to the point where I had to install a polyfill to enable drag and drop on mobile). On desktop the divider is on top, on mobile is on the bottom. Also, because lichess is not really particular about how the website looks on mobile, you will probably not be happy unless you also enable the Mobile theme. Tests with various devices show wildly different results. On my phone and the Chrome emulator it works smoothly, on a friend's Motorola the resize bar doesn't seem to work properly. So be warned, mobile support is (and will probably remain) sketchy.
  • Turning the feature off and on again will also reset the heights, if you get in some sort of trouble.

Sound Options

category: General - Advanced
values: No move sounds - defaults to none

This feature gives finer options for sound. At the moment it has just one setting, to disable piece movement sounds only. Note that if you have set the Silent sound schema or the volume to 0 you will not hear anything anyway. 

Sound volume (0-100)

category: General
values: Number between 0 - 100 - defaults to 70

Self explanatory. It sets the volume of the sounds generated by LiChess Tools.

Time alert (minutes)

category: Play - Advanced
values: 0:30 / 1:00 / 1:30 / 2:00 / 3:00 / 5:00 / Sound alert- defaults to none

This feature allows you to choose additional time warnings during play. Every time an enabled threshold is reached, the bottom player name and clock section will glow red and, if so enabled, a low time sound will be emitted.

Various TV options

category: TV
values: Link for current TV game / Bookmark for current TV game / Streamers current games / Friends current games / Previous two games in player TV / Prevent screen lock with TV / Persistent TV category- defaults to all

This feature has been introduced in version 2.2.0 to merge four existing tools for TV with the same functionality. Make sure you review your Preferences.

Options:

  • Link for current TV game - adds a link to the currently played game in TV on the title, just like in Analysis mode
  • Bookmark for current TV game - adds the ability to bookmark a currently playing game by hovering the mouse next to the game title
  • Streamers current games - adds a Streamers tab in the Current Games page where you can see the games of all currently playing streamers
  • Friends current games - adds a Friends tab in the Current Games page where you can see the games of all currently playing players that you follow
  • Previous two games in player TV  - In category TV (when you are watching Blitz games for example) you get a list of two previous games in that category under the main playing board. For player TV, when you go to a specific player to watch their games, this is missing. This feature will add the two previously played games by the player you are watching.
  • Prevent screen lock with TV - will prevent screen locking on mobiles or other such devices while you are on a TV page.
  • Persistent TV category - will remember which "channel" you watched on (best, blitz, classical, etc.) and will select it again when you go to Watch -> Lichess TV.

Previously viewed game menu item

category: TV - Advanced
values: yes / no - defaults to yes

  • This feature adds an extra menu item in the Watch menu that will take you to the previously seen chess game. Just opening any game will consider it as watched. If you go to the previously watched game, then clicking on the menu item again will take you to the game you watched before that, and so on. The total number of remembered games is 10.

Pin broadcasts and studies to home page

category: General- Advanced
values: yes / no - defaults to yes

This feature will add a red pin to the studies and broadcasts tabs on the top left, right after the search icon. Click on this pin to pin/unpin the study or broadcast to the home page. All the things you pinned will be added to the spotlights area of the main page (top left corner)

Last visited study menu

category: Study - Advanced
values: yes / no - defaults to yes

This feature adds an extra menu item in the Learn menu that will take you to the previously visited study.

Hovering over the entry will show a maximum of 5 previously visited studies. This feature does not work for mobile devices.

Just opening any study will consider it as visited.

Notes:

  • the Broadcasts feature and some Puzzles use studies in the background, so this will show them as the last visited study

Styling for study comments

category: Study
values: yes / no - defaults to yes
Needs log in

This feature adds the ability to style the comments in your PGNs. You can choose between a number of styles that you can place anywhere in a comment with cls:<styleName>. It also adds a new button next to the comment button in a study chapter (a quill in an inkwell ) that with cycle through all of the classes at the very beginning of the comment to the current move. The usable classes are: red, orange, yellow, green, lightgreen, cyan, lightblue, blue, violet, magenta, pink, underline, strikethrough, italic, bold, cursive and nothing or clear to clear existing class. (ex: cls:red will make the rest of line of the comment red)

Notes:

  • the name of the class declares the intent, not the actual result of the styling. Given that there are dark and light themes on lichess, the styles have been updated to be readable in both situations, using shadows or slightly different colors.
  • the style of the comment will only be visible to you and people with LiChess Tools installed. All others will see cls:blue or whatever in the comment.
  • you can actually put anything as the class name and the comment will be put inside a span with the class lichessTools-<your class name>, which allows you to customize your style even more with the addition of custom styles from other extensions like Stylus. In this case all people seeing the comment would also need the custom styling.

Show the order of arrows and circles

category: Analysis/Study - Advanced
values: yes / no - defaults to no

This feature shows a number on each drawn shape, so that you can see their order. Sometimes in studies you want to convey the moves that could follow the current position using arrows and circles, but in certain situations the order of the moves is important. Set this to true in order to see which arrow comes first.

Ctrl-Space for best computer move

category: Analysis/Study - Advanced
values: yes / no - defaults to yes

The default functionality of the Space bar in analysis mode is to start the computer engine if not started or play the best move in the computer move list. Because I was often using lichess with YouTube in another tab, pressing Space accidentally was always an annoyance. This feature changes the shortcut to Ctrl-Space.

Notes:

  • A previous version of the feature was using Shift-Space, but it would overlap with a lichess feature.

Clear chapter artifacts

category: Study - Advanced
values: yes / no - defaults to yes

This feature allows you to choose which artifacts to clear from the current study chapter only. To use it, click on the cog button next to the chapter name and, instead of the Clear annotations button now you get the choice of just comments, glyphs (good move, bad move, that kind of thing) and drawn shapes (circles and arrows) as well as all of the above. There is an extra option of clearing all PGN tags. This feature also adds individual delete buttons to all tags in the current study chapter.

Notes:

  • this only works for the current chapter, as the site does not have access to the moves or tags in the PGNs of other chapters
    • the feature could be modified to work on all chapters, but it would have to first select the chapter, then alter it, which I think is not something that should be done automatically.
  • this works on the client, which means that instead of sending one command to the lichess site, it creates delete requests for each item. If the PGN is large, this can take a long time.

Customize chat buttons

category: Play
values: yes / no - defaults to no
Needs log in

When playing the first few moves and after the game has ended, lichess gives you the option to click on some buttons for some standardized message like Hello, or Have fun! This feature will give you the ability to customize these buttons.

The interface may not be the most intuitive. To edit buttons you click on the little cog button, to exit edit mode you click on the same. This allows altering, adding and deleting buttons. Click on the small X button to enter delete mode, after which you click on buttons to delete them. Delete them all and you get the default buttons back.

In order to add or alter buttons you have to write in the chat text input something like <name>/<message>. Example: hi/Why, hello there, old chap!, which will create a button with the text HI which will send the chat text "Why, hello there, old chap!". In order to add the button, you click the + button. In order to alter an existing button, you click on that button.

Notes:

  • the name of the button can be only 2,3 or 4 characters long.
  • if you attempt to add or edit a button without a recognizably correct input text, the input box will jiggle to let you know you are doing something wrong
  • to edit the end of game buttons you have to reach the end of a game. This may be inconvenient and I am thinking of solutions
  • I am considering expanding this feature with buttons for the duration of the game as well as for spectators. Let me know what you think!

Player lag indicators

category: Play
values: none / bars / chart - defaults to none
Needs log in

This feature adds bars or a chart next to your and your opponent's name showing the lag and server latency. To limit use of server resources, the opponent lag indicator is updated only every 5 seconds, while yours every second or so.

Player warning alert

category: Play - Advanced
values: yes / no - defaults to no
Needs log in

This feature will add a small warning icon next to the opponent's name if the percentage of disconnects for the current time control is higher than 3%. Other warning reasons may be added in the future. The purpose of this is to allow you to abort a game before wasting time on a malicious opponent.

Note: this is just an informative alert, you do what you want with it. If you abort too many games, you might end up getting punished by lichess.

Common teams

category: Play/Analysis/ Study - Advanced
values: yes / no - defaults to yes

This feature will add a small icon in the player crosstable (the thing showing you how many times the players have played against each other and the results of the latest games) if the two players in the game share at least one team. Mouse over the icon to see which teams. Clicking on the link will open the first team page. 

Play layout

category: Play
values: Normal / Hide left side / Hide chat - defaults to Normal

This feature will control the layout of the page when playing.

Options:

  • Normal - no changes
  • Hide Left side - will hide completely the left side of the screen (game information and chat) in large screen layouts (width 1260px or more)
  • Hide chat - will hide the chat and rotate the game information, gaining some space in large screen layouts (width 1260px or more)
    • in this mode, the game icon in the game information section will glow if there are unread chat messages. Clicking on it will toggle the chat being shown/hidden and mark existing chat messages as read.
  • Option to hide chat - same as Hide chat, but chat starts by being shown and you have the option to click on the game icon to toggle the chat.

Timeline notifications

category: General - advanced
values: Forum post / Blog post / Lichess announcement / Stream start / Simul create / Simul join / Team create / Team join / Tournament join / Following / Study like / Blog post like - defaults to Forum post, Blog post
Needs log in

This feature will add a notification when there are unread items in your Timeline. The default values include comments on forum or blog posts you follow, forum or blog posts from people you follow. Basically posts that you wouldn't normally be notified about. 

Clicking on the notification will take you to the Timeline page. Opening it from here or anywhere else will consider the items read and so you will not see the notification until new ones turn up.

Notes:

  • there will be no notification if quiet mode is on (meaning you are playing a game or you have quiet mode manually set)

Mobile device features

category: General
values: Evaluation gauge / Hide the octopus mascot / Analysis arrows / Random move button / ... only when variations /Scroll lock when playing - defaults to Evaluation gauge, Random move button, ... only when variations 

This feature alters some behavior specifically for mobile devices. There are Chromium-based mobile browsers, like Kiwi, which accept browser extensions. Enjoy LiChess Tools on mobile installing one and then the extension, then customize the experience with this feature.

The options are:

  • Evaluation gauge - makes the evaluation gauge visible even on small screen widths (for mobile)
  • Hide the octopus mascot - hides the octopus mascot that takes a lot of space in Interactive Lessons
  • Analysis arrows - adds a button next to the Explorer and Practice ones that allows for drawing arrows and circles on mobile devices in Analysis board and Studies
  • Random move button - adds a button between the previous and next move buttons that will randomly play a move from the move list (same functionality as Ctrl-right to play random next move from list on desktop)
  • ... only when variations - this makes the Random move button above only appear when there are variations to choose from
  • Screen lock when playing - will lock scrolling and zooming on mobiles when playing (or during puzzles). A lock icon will appear as a button on top of the screen that you can tap to temporarily lock/unlock the scrolling on that page (refreshing or going to other screens will again automatically lock the screen unless you unset the value from Preferences). Whether the screen is locked or not will be persisted, so you have to tap it again to enable if you disabled it.

Notes:

  • Because both the shape drawing and random move buttons take up space, this feature also changes the style of the buttons so that they take two rows of space: the first for utility buttons (left) and the hamburger menu button (right) and the second row for PGN navigation buttons (first, previous, random, next, last move). If none of the two options mentioned are enabled, then the regular "one row for all buttons" style is used.
  • Screen locking when "playing" doesn't necessarily means when playing, but when lichess is in "playing mode". For example that also means when you haven't yet started the game or when you just ended the game or when you are doing puzzles.

Mobile device game features

category: General - Advanced
values: Game arrows / Standard buttons / Swap user and clock - defaults to none

This feature alters some behavior specifically for mobile devices when in-game. This means playing, or solving puzzles or even watching running TV games. There are Chromium-based mobile browsers, like Kiwi, which accept browser extensions. Enjoy LiChess Tools on mobile installing one and then the extension, then customize the experience with this feature.

The options are:

  • Game arrows - adds a button next to the button for Analysis that allows for drawing arrows and circles on mobile devices in games
  • Standard buttons - the default behavior of lichess.org is to hide the move navigation buttons for games on small screens, instead showing one line of PGN with previous and next move buttons on either side. Set this in order to see all buttons (including the Game arrows one above). For normal play this is not necessary, but in puzzles, TV and such screens this is the only way to access the drawing arrows button.
  • Swap user and clock - swaps the position of user and clock during play, to avoid hiding the clock with your dominant finger.

Colors for shapes on mobile

category: General - Advanced
values: 1 / 2 / 3 / 4 - defaults to 1

This is part of the Mobile Experience feature and it selects the count of colors you can use on mobile to draw arrows and circles. If you only want to turn the drawing mode on/off with a single tap, select 1.

Show evaluation of explorer moves

category: Analysis/Study - Advanced
values: From computer eval / From ChessDb / From Lichess / From winning stats / Rows from eval / Hidden - defaults to From computer eval, From ChessDb

Options:

  • From computer eval - will show the evaluation from the computer engine
  • From ChessDb - will show the evaluation from the chessdb.cn database
  • From Lichess - will show the evaluation from the Lichess evaluation API
  • From winning stats - will compute an evaluation based on the win/draw/loss statistics
  • Rows from eval - will add extra rows with the move and the evaluation if the explorer doesn't contain a line for that move
  • Bar precision - will add a decimal digit of precision to the percentages in the move Explorer bars
  • Hidden - hides any evaluation in explorer, but retains the settings if you want to show them again. This option can also be changed from the Explorer settings

This feature will show the move evaluation for Explorer moves. This information is retrieved from three different sources that can be enabled or disabled: the local computer eval, chessDb.cn, the lichess evaluation API and the Explorer stats themselves. If Hidden is set, then the settings for evaluation will remain, but the column will not be shown. The feature can be hidden not only from Preferences, but also from the Explorer config screen.

ChessDb.cn will be used first (if enabled) and then the Lichess eval API, so if you prefer the Lichess API, you should disable ChessDb. In case ChessDb fails for whatever reason, lichess eval API will be used instead. Just note that the Lichess API is limited in how much calls one can make and contains less data than ChessDb.cn.

The statistics eval will only be shown if there are 100 or more games in the lichess database for that move and not all won by one side. Its faded coloring signifies that it's not a true eval, but if enabled can compare with the other values. 

Computer evaluations will have a white color, stats evaluations will have a grey faded color and the cloud evaluations will be slightly blue. If using chessDb, the eval will show either red, green or bright green for bad, good and best moves.

This feature also adds a warning icon on the right side of moves that have large differences between evaluation and winning stats, indicating a possible trap or gambit situation. It also calculates the sharpness of a move, as defined in Evaluating sharpness with Leela’s WDL and adds a blue warning icon on the right side of the move when it's higher than 100.

The feature is unavailable in the Analysis screen of a running correspondence game.

Notes:

  • The more lines you have configured on your computer analysis, the more items in the list will be evaluated. Explorer can show 12 moves, for example, while the computer analysis is configured with Multiple lines=2, this means a maximum of 2 items in the Explorer will have an evaluation. With 5, you get 5, assuming the computer moves are in the Explorer list.
  • There is an internal cache of these evaluations, so you can run the computer eval for some moves, then stop it, but when going to those moves you will still see the evaluations as done before at whatever depth was last computed.
  • Take care with stats evals in positions with few games. It might tell you that you are absolutely winning, but that's just because no one played a winning move from that position. That is why eval from winning stats is disabled by default.
  • The evaluations from cloud differ in depth based on how many moves you want to see. For example you could ask for just one move (the best in the position) and it might show you the eval with depth 55, but if you ask for two moves it will give you depth 44, for 20 moves you would get depth 13 or even nothing. To account for this, cloud eval combines the values from two calls: for 5 moves and for 10 moves. Since the cloud data comes from local evals, server evals and different versions of Stockfish and I am combining two different depth evals, take the evaluation as an estimation, not a given.
  • Changing the options for evaluation in Preferences will reset the local cache for each move.
  • Lichess' cloud eval API will return 404 when no data is available, resulting in ugly red network lines and error messages in the browser developer tools. Those are not errors, but there is no way to hide them in Chromium browsers from JavaScript. If it annoys you, you can turn them off from DevTools (see https://stackoverflow.com/questions/4500741/suppress-chrome-failed-to-load-resource-messages-in-console)
  • Depending on how much you do analysis and how overloaded the servers are, cloud eval might start returning 429 errors, meaning the servers report you are abusing them. A warning will appear every minute while that happens. The workaround is to disable From Lichess and use just computer eval (which, of course, will also use cloud eval :-P). The configuration on lichess' side, at least for the moment, is 3000 calls per IP address per day. That's a bit low, but it might be enough.
  • If computer evaluation is started and used as a data source, the cloud eval will not be accessed.

Show explorer moves leading to gambits

category: Analysis/Study - Advanced
values: yes / no - defaults to no

This feature adds a new column to the Explorer table showing the number of gambits reachable from the current position and for each move to play. The feature can be turned on/off not only from Preferences, but also from the Explorer config screen.

The feature is unavailable in the Analysis screen of a running correspondence game.

Options for puzzles

category: Puzzles - Advanced
values: Prevent screen lock playing puzzles - defaults to Prevent screen lock playing puzzles

This features controls various options related to puzzles.

Prevent screen lock playing puzzles - this will prevent the screen from autolocking when playing puzzles (or thinking about how to solve them).

Puzzle performance chart in Profile

category: Puzzles - Advanced
values: yes / no - defaults to yes
Needs log in

This feature will add a new Puzzles entry in the Profile, right under the ones for various time controls (blitz, classical, etc.). The top part of the page will show a chart and a time slider to select the period for the statistics.

If the slider end is the same as for the selected interval, the bottom part will list the puzzle themes and the performance for each, as well as links to the failed puzzles that need replaying.

A new entry in the Puzzle menu called Puzzle stats will lead to the same page.

Notes:

  • if you have enabled Profile slider options, all of the settings there will apply to this slider as well.
  • if the selected time interval does not end at the end of the entire slider period, the bottom statistics will not be shown, as the data for them is only extracted relative to the current date. In fact, the end date of the slider might be different from today, so the stats are taken from the beginning of the interval to today.
  • the green background bars show the puzzles you completed, with a darker gradient towards the puzzles you completed on replay. Mouse over to see the actual values of wins + replay wins.
  • click on the theme name to play puzzles in that theme. click on the To replay numbers to replay the puzzles in that category.
  • The links to replaying puzzles are formed using the number of days you've selected in the time slider, but on the server side (which I don't control) that number is limited to one in the following list: 1, 2, 3, 7, 10, 14, 21, 30, 60, 90. So let's say you move the slider to 2057 days into the past, it will actually serve you the puzzles in the last 90 days. Same for 76. Unfortunately that is a Lichess limitation, designed to not overload their servers, so nothing to be done there.
  • Warning: based on which theme you choose, you might find that you lose a lot when you fail and win very little when you win, or vice versa. Truly, the idea of a general puzzle rating makes no sense and you should not consider it. Instead, look at the performance value for each theme. Do not let your chess progress get hampered by a meaningless numerical value.

Find position in Lichess puzzles

category: Puzzles/Analysis/Study - Advanced
values: a NIF file

Work in progress! - this means use at own risk.

This feature requires a separate file to be downloaded on your computer. Once loaded, the Explorer will try to find puzzles similar to the current position. Since there are more than four million Lichess puzzles, the only way to create a small enough index on your local machine was to just keep the significant elements of the position FEN, meaning that the positions will not be exactly the same. In v2.3.172 I've added a CRC mechanism to dramatically reduce false positives. 

Get the latest file here: puzzle.nif.zip (about 330MB) - download it and extract it somewhere, then pick the file in the Preferences.

So it works like this: in analysis, if the current position is found in Lichess puzzles, a new section will be added to Explorer called Puzzles, with a list of puzzle links. When this happens, also a new tab is added to Explorer with a capital P on it. You click it and Explorer will scroll down to the Puzzles section.

Download PGN for puzzles

category: Puzzles - Advanced
values: yes / no - defaults to yes

This feature will add a button next to the puzzle id (once the puzzle has been completed or abandoned) to download the puzzle PGN. A popup with the PGN will appear and you can copy whatever part of it you want. The PGN will start with a FEN position and the puzzle moves.

Quiet mode on all tabs

category: Play - Advanced
values: yes / no - defaults to yes

When a game starts, lichess sets a quietMode variable to true, instructing various notifications to not be displayed. Unfortunately, that only happens on the page that one plays on. This feature makes all open lichess pages get the value for quiet mode from the same place, so they don't behave inconsistently.

There is also a button added to the lichess menu which can manually enable/disable quiet mode. Warning: once manually enabled, quiet mode will remain on until manually disabled! The normal game playing quiet mode will turn on and off automatically, as before, only it will affect all open lichess pages.

Note:

Insert new chapter after current one

category: Study - Advanced
values: yes / no - defaults to yes
Needs log in

This feature adds a button to the create study chapter form with the text Create after current. Using this button instead of the normal Create chapter will create the chapter immediately after the currently selected chapter.

Note: the button will not appear if already on the last chapter in the study.

Force add Stockfish 16+

category: Analysis/Study - Advanced
values: yes / no - defaults to no

Some browsers have issues with the new Stockfish 16 engine and above, but these issues are not clearly defined. For example many users of the Brave browser complained of division by zero errors. I use Brave, I've never seen this. Therefore, if you are a user of Brave or some other browser that does not support the Stockfish 16 engine or higher, this will force it to be available. However, the responsibility of enabling this and using the engine are yours.

Don't worry, there is nothing bad that can happen, you just switch to Stockfish 14 and you're done in case this doesn't seem to work.

Show PGN in studies

category: Study - Advanced
values: yes / no - defaults to yes

This feature adds a textarea in the study Share tab called PGN. Similar to the one in Analysis board, it will show the PGN text of the current chapter. The PGN will be generated by the same code that generates the one from Extra context menu options and may differ from the one generated by the Copy PGN or Download PGN buttons.

Persist study settings

category: Study - Advanced
values: New/edit chapter settings / Position/move in the study - defaults to Position/move in the study
Needs log in

Options

  • New/edit chapter settings - will persist the settings for studies when you create or edit them. Then it will use the same values when creating new studies.
  • Position/move in the study - will remember the position (move in the list) in a study chapter and restore it on page refresh
    • Note that if lichess does not restore the chapter on refresh (it happens sometimes when the user get disconnected) then this will not work. This just restores the move in the current chapter, it does not change it.
    • Also note that this will not work in Interactive chapters while playing them 

Move list options

category: Analysis/Study - Advanced
values: Indented variations / Bookmarks / Expanded move list / Hide left side / Open in new window / Eval button on the right - defaults to Bookmarks

This feature will change the functionality or appearance of the move list in Analysis board. There are already such changes with the Highlight moves in analysis functionality.

Options:

  • Indented variations - this will make even variations that follow inline (when they are less than three, they are displayed one after the other in parentheses, like in a PGN text) show as tree branches, increasing readability and preparing for new features that will use this.
  • Bookmarks - Study only - this will allow for bookmarks, which are very cool (read below). Let me know how they could be more of use to you.
  • Expanded move list - Only for desktop resolutions (1260px+), this feature is removing the left and right margins and expanding the move list to the edge of the screen and also downwards. It is meant to improve analysis on very complex PGNs.
  • Hide left side - Most of the time the left side of the analysis screen is useless: chat, notes, study participants and stuff like that. With this, you can hide that side to gain even more space for the move list. This is intended to be used together with Expanded move list, but you can use it separately as well. 
  • Open in new window - Study only - this adds a small button in the top-right corner of the page, just before the computer evaluation toggle. Clicking it opens a new window with just the analysis tools (move list, computer eval, explorer) that you can move to another monitor, for example, or resize and place wherever you want. Use the study SYNC button to keep the two windows synchronized. (the SYNC button only appears under the board when the study is set to Enable sync)
    • this is also useful for printing PGNs, as the new window is optimized for printing.
  • Eval button on the right - this will move the computer evaluation button to the right side of the eval header and also make it a little bit smaller.

How bookmarks work:

  • right-click on a move in a study move list and select Add/Remove bookmark from the context menu
  • put any text in the textbox that appears (or remove it all to delete the bookmark)
  • this will create a bookmark, which serves several purposes:
    • it will show as a named label in the move list (good for naming variations or making the PGN more readable)
    • it will allow to expand/collapse the branch from that move on (which will also persist between reloads)
    • it will serve as an anchor for URLs to this exact variation
  • for any bookmarked move you can right-click it and get:
    • Collapse/Expand all bookmarks - collapse and expand all bookmarks in the tree
    • Get bookmark URL - will copy to clipboard a URL to that specific move in that specific chapter and study (the link will look like this: https://lichess.org/study/<studyId>/<chapterId>#<bookmark name>)
    • Split chapter here - only if the bookmarked node has children (following moves) - it will create a new chapter with the following moves, then add a comment with the URL to the new chapter. If you want to also delete the following moves, press Shift when you click on the menu item. The confirmation dialog should reflect that moves will be deleted.

Notes:

  • Bookmarks are saved in the comments as bkm:<label>
  • Only you and people with the LiChess Tools extension (and bookmarks enabled) can see and use the bookmarks, the rest will just see the bkm:<label> in the comment
  • Splitting a chapter with Shift-click DELETES THE EXISTING FOLLOWING MOVES from the initial chapter. Don't tell me you weren't warned.
  • You cannot collapse a bookmark if the current selected move is under it
  • selecting or moving to a position under a collapsed bookmark will expand it

Link to download all studies of a player

category: Analysis/Study - Advanced
values: yes / no - defaults to yes

This feature will add a "Download all studies" link in the studies list section whenever a user is selected in the query. This happens in two situations: either you went to your studies or you selected the studies of another player. The necessary condition for the link to appear is that the study search query contains "owner:<user>" where <user> is any user id. The link will download one PGN with ALL studies of the selected user that you have access to. Naturally, your studies will all be downloaded. For other users, only the public studies will be downloaded.

Study flairs

category: Study - Advanced
values: Author flair / Member flairs / Flairs from study topics - defaults to Flairs from study topics

Important: Lichess has implemented the same feature for study and study owner flairs. If you want to stick with the native version, turn all options off here!

This feature allows adding flairs to studies. The first two options add the flairs from the author and/or members of the study. The third one allows adding your own flairs to the study, using the study topics.

Flairs will appear in this order:

  • study flair - as chosen by the native study flair (only if the topic flairs is on)
  • study topics - as chosen in the study topics (only if the topic flairs is on)
  • study owner (only if the author flair is on)
  • study members (only if the member flairs is on)

Example: you want to add an alien flair to your study. The flair name is smileys.alien, so your topic needs to be flair.smileys.alien. Don't worry, all you have to do is go to your study, PGN tags, select Manage topics then type alien and the smiley will appear in the dropdown ready to select.

An even simpler solution is to press the Flairs button on the left of the popup which gives you a normal flair picker to select with.

The result is that, in the studies list, instead of the four chess squares icon for all studies you will get the first study flair icon, and the rest of the flairs will be listed on the bottom of the study. The flairs of the study come first, then the one of the author, then the one of the invited members (if the options are set to show them).

The list of all possible flairs can be found at https://lichess1.org/assets/flair/index.html

Notes:

  • user flairs will show the user powertip on mouse over
  • topic flairs will go to the topic search on right click. This is to avoid confusion for people who just want to open the study. On mobile this works with long click.
  • for people not having LiChess Tools installed, there will be no flairs in the study lists and when opening the study they will see the flair as text (ex: flair.smileys.alien)

Mirror button in Board Editor

category: Board Editor - Advanced
values: yes / no - defaults to yes

This will add a new button to the Board Editor () which will mirror the position you have currently. Unlike Flip board, it will create the exact same position for the other side, with the exact same valuation and move opportunities.

Custom mini-game size

category: General - Advanced
values: a number - defaults to unset

This feature sets a custom size to mini games, which are the boards that appear when you hover over a playing player link or in the Current Game or Broadcasts section, etc. The default is unset, which keeps the native lichess style. An equivalent numerical value for the default would be around 20. Change the number to increase or decrease the size of mini boards.

Notes:

  • the minimum value that will be taken into account for mini-games is 17. Lower values will only affect the popup board in the computer eval window. That's because we have to take into account the ratings and username text in the user info popups.

Remove chat link warning

category: General - Advanced
values: yes / no - defaults to yes
Needs log in

An annoying "feature" in lichess is asking (randomly and inconsistently) if you want to leave the Lichess web site when clicking on a link in the inbox chat. This tool removes that warning.

Lobby page elements

category: General - Advanced
values: Side / Play grid / Play buttons / TV / Blog / Daily puzzle / Support / Feed / Tournaments / Leaderboard / Winners / About - defaults to all

This feature selectively hides/shows elements in the lobby. Customize the main Lichess page to your heart's desire.

If the Play grid is hidden, then the link to Play will go to a page that shows just the play grid and buttons - this only happens in desktop mode, because on mobile there is just one link to the main page, not two.

Notes:

  • At the moment the elements will just be hidden, the space they take remains there, unless an entire row of elements is hidden. Making every possible combination of elements arrange correctly is not trivial.

Daily quote

category: General - Advanced
values: yes / no - defaults to yes

This feature will show a daily chess quote in your lobby on large enough screens. You can close it, which means you will not see it again that day.

Timeline in Profile

category: General - Advanced
values: yes / no - defaults to yes
Needs log in

This feature will add a new tab to your profile page with the timeline items. The same can be achieved by clicking on the "more" link at the end of the lobby timeline or if you enabled timeline notifications and you click on the notification. However, since the lobby feature can remove that part of the page and you may not want to enable notifications, here is a simple way to get to that page without any issues.

Fix board coordinate position

category: Analysis/Study - Advanced
values: Fix outside coordinates / Larger coordinate font / On each square - defaults to Fix outside coordinates

This feature has two purposes. The first is to fix a Lichess bug that doesn't apply the preference for board coordinates on the outside in Analysis/Study. The second is to fix the CSS a little bit and also to enlarge the coordinates font.

Since most people found the larger coordinate font ugly, you have to enable the larger font manually and is disabled by default.

Also, a third option that doesn't exist natively on Lichess will put the coordinates On each square of the board. (Lichess also implemented this, so you get a choice on what to use)

Profile slider options

category: General - Advanced
values: Show dates / Add 1W filter / Fix small intervals - defaults to all

On the profile of a user there are buttons to select the range of the chart on top of the page. There is also a slider that can select custom ranges and one can also drag the selection on the chart itself. This feature improves the way these elements work:

  • Show dates - will show the start/end date text in a label
  • Add 1W filter - will add a one week filter button
  • Fix small intervals - will update the slider minimum and maximum values in order to be able to be able to control small time intervals. When the start and/or end of the slider is unbounded (no left/right border) it means that there is more on that side which can become accessible if you increase the size of the interval or if you pan the slider in that direction. Having extra information to the left or right of the slider will change the text of the label to orange accordingly (start date, end date or both).

Game list options

category: General - Advanced
values: Filters / Selection - defaults to all

This feature handles options for list of chess games, like in Profile or Advanced Search. The options will be stored locally so they persist on page reloads

Options:

  • Filter aborted - a filter to show/hide aborted games. Aborted games are considered games where one side has not moved.
    • Unfortunately that also includes currently playing games where no one has moved yet.
  • Filter analysed - a filter to show/hide games with a computer analysis
    • Warning: if you have a lot of games and very few with computer analysis, this will hide all of the games without analysis, leading to loading many pages of the game list really fast. Lichess might report a server overload.
  • Filter titled opponents - a filter to show/hide games against a titled opponent.
    • an opponent is considered a player that is not the one on who's profile you are on. If the page is not a profile page, like Advanced Search, then an opponent is a player that is not you.
    • Warning: if you have a lot of games and very few with titled opponents, this will hide all of the games without titled opponents, leading to loading many pages of the game list really fast. Lichess might report a server overload.
  • Color by players - will change the background color of each game based on the player names. The same two players, regardless of the side they play, should generate the same background color. This helps highlight matches.
  • Go direct to analysis - will make clicking on a game row take you directly to its analysis page, bypassing the game page.
  • Selection - this will add a checkbox to every game. Once at least one is selected, a copy button will appear at the top of the list which will copy all the PGNs of the selected games to the clipboard.

More decimals for computer evaluation

category: Analysis/Study - Advanced
values: yes / no - defaults to no

If you enable this feature, the computer evaluation will show two decimals instead of one for computer evaluation values. This applies to the main evaluation value, the evaluation of each point of view as well as the evaluations of individual moves in the move list (if available).

Active title icon

category: General - Advanced
values: yes / no - defaults to no

This feature will replace the default lichess tab icon in your browser with the lichess flair icon by default - which I believe is much nicer - and then change it depending on various conditions.

If the tab contains a game (whether watching TV, a broadcast, a study or a game, playing or running puzzles) and the game is not ended, then the icon will change to a black or white pawn, depending on whose turn it is.

If you need more icons for other situations, let me know. So far, the reason I added this tool is to easily see if someone moved in the game I was watching or if the game has ended or a new one started.

Blog editing options

category: General - Advanced
values: Auto save / Save button - defaults to Save button
Needs log in

Options:

  • Auto save will automatically save the blog you are editing every 30 seconds. This will only happen if the content of the blog has changed from last save and it is still in draft.
  • Save button will add a Save button 😁. This will allow you save the blog you are editing and continue working on it.

Computer evaluation line options

category: Analysis/Study - Advanced
values: Highlight same moves - defaults to none

Options:

  • Highlight same moves - this will colorize the moves if found several times in the computer eval lines. Each specific move will have a different color, so you can easily determine the difference between moves.

Go to Analysis on game end

category: Play
values: loss / draw / win - defaults to none
Needs log in

This feature will automatically open the Analysis page when your game ends with the selected outcome. Note that draw means anything that is not a win or a loss, so even aborted games.

Themes

category: General - Advanced
values: whatever themes are available - defaults to none

This feature will enable/disable various CSS themes. If an external theme (made by someone other than me) it will show attribution in the Preferences page. It will also be the creator's responsibility to maintain it.

Available themes:

  • Performance - performance CSS changes, mainly attempting to remove all animations, which cause slow rendering
  • Just Explorer - will hide the computer eval moves (but not the arrows, if enabled) and the move list when the Explorer is open. This will only be applied for mobile devices
  • Mobile - a lot of style improvements for mobile, especially aimed at studies and interactive lessons
  • Slim arrows - will make arrows less thick: color arrows, variation arrows and computer evaluation arrows.
  • ... slimmer - only with Slim arrows enabled, makes the arrows even slimmer
  • Less icons - will hide the header lichess and logged user icons
  • No sticky header - will make the page header scroll with the page
  • No study chat - eliminate the chat from studies and expand the chapter list for large screen layouts.
  • Nicer piece drag - adds some sizing and borders to piece dragging, enhancing the experience
  • No grab cursor - the default lichess functionality is to have a hand mouse cursor when hovering over a piece and a grabbing cursor when grabbing it. Due to popular demand, this theme reverts that behavior and only uses the default arrow mouse cursor for pieces.
    • lichess reverted the change, so there is no grab cursor anyway. I leave this here for historical purposes, but might be removed in the future.
  • No Practice button - especially useful on mobile devices, it hides the Practice with computer button under the analysis tools, thus preventing you from starting analysis for no good reason by mistake
  • Flexible game move list - the move list in games (so when playing or watching games on Lichess TV) is pretty limited in space. This theme fixes that
  • Thick analysis gauge - the analysis gauge will be 50px thick

Notes:

  • Multiple themes can be applied at the same time, but beware of conflicts. 

Wiki pages based on FEN

category: Analysis board - Advanced
values: yes / no - defaults to yes

This feature will show Wiki pages on openings even if the order of the moves changes. Basically, it looks for the Wikibooks Chess Opening page associated with the current position.

OBS Integration

category: Integration
values: yes / no - defaults to no

This feature will add an OBS button to broadcasts. There you can set up mappings between broadcast boards/chapters and OBS scenes, so that changing the board selection will change the scene as well. Each broadcast has its own setup and mappings.

Don't forget to enable the WebSocket server in OBS' WebSocket Server Settings

The default OBS connection settings are localhost, port 4455, no password. If you want to use passwords or some other URL, you will have to open the OBS setup popup by pressing the button (you will get no scenes to map), then update the password and/or URL, save the settings, then reopen the OBS setup. Now, with the correct password, the OBS scenes should be available. But the simplest thing is to not use a password and run the server locally on the default port.

By default, broadcasts will have OBS disabled and the OBS button for them will appear dimmed. Click on it to set up the mappings and connection properties and save to enable it. You can also enable/disable the OBS integration for the broadcast by right-clicking on the OBS button or by pressing O.

There is a default scene mapping that, when set, will change the scene for all boards that don't have a specific mapping. This scene will also be used for the board list view of a broadcast (no big board displayed, just the list). If you don't want this functionality, set the default scene to empty.

Unselect piece after specified seconds

category: Play - Advanced
values: numeric- defaults to empty

If you select a piece while playing (games or puzzles) and wait for the number of seconds specified by this setting, the piece will get automatically unselected. This only works for the last move in the list, so for example this will stop working after you make a mistake in a puzzle and there is a wrong move after the current position.

Easy access full screen button

category: General - Advanced
values: yes / no - defaults to no

If enabled, this will hide the Lichess header in full screen mode and will add a top header button () that will enable a full screen mode when pressed. To return to normal press Esc, F11 or move the mouse to the top of the screen and click the hovering X button.

Notes:

  • this only works in Desktop mode, not for mobile.
  • in Zen mode the full screen button will be visible. If the option is on, full screen in Zen mode will not show the Lichess logo and the button to exit Zen mode, therefore if you want to exit Zen mode you either use the Z key shortcut or exit full screen mode, then use the exit Zen mode button.

One click move

category: Analysis/Study - Advanced
values: Analysis/Study / Only orientation side / Move from PGN - defaults to none

This feature enables one click moves, when clicking on a square where only one piece can legally move will perform that move.

Options:

  • Analysis/Study - this enables one click move in Analysis or Studies
  • Only orientation side - this forces the feature to only work for the side that the board is oriented for. So if you see the board from the White orientation, it will only work for when it's White's turn to move.
  • Move from PGN - this is only for the situation where the square you clicked on can accept multiple pieces moving there. In that situation, it will only work if you are not in Interactive mode or you can write to the study (owner or write access contributor). Then if the PGN (analysis or study) contains only one move that reaches the destination square, it will make that one.
    • this is mostly an accessibility feature to help easily do training without clicking and moving the mouse a lot.
    • when it's ambiguous what the player meant with a click, the pieces that can move to that square will flash briefly. This also will happen when there is only one move in the study you own and that move will be made.

Notes:

  • this only works in Analysis/Study for now. Altering the way the game is played is against the Lichess Terms of Service, but I am working on convincing them to implement it natively.
  • to clarify the Move from PGN option, it will not apply in the following situations:
    • you are in interactive mode and you are not an owner or writing contributor - the idea here is that if you can always exit interactive/preview mode you can find out what the next move is, but if you cannot, you don't know what the next move should be
    • there are moves in the PGN that reach the clicked square, but there are multiple ones - LiChess Tools doesn't know which one to pick
    • obviously, if you have access to the PGN you can just look what the move is, so there is no conflict in non interactive mode or in the Analysis board
    • running correspondence games

Mouse wheel during game play

category: Play
values: yes / no - defaults to no

This feature will enable mouse wheel during play. Just like in Analysis, it will move to the previous/next move in the move list. This behavior is disabled by default in Lichess as to not interfere with game play, so that's why it's also not enabled by default in LiChess Tools.

Expand all variations

category: Analysis/Study - Advanced
values: Show button / Auto expand - defaults to false

Options:

  • Show button - adds a button looking like a magnifying glass with a plus sign in in () in front of the first move in the move list tree in case any of the variations in the tree are collapsed. Clicking on the button will expand all variations recursively and then hide the button. Any operation that will result in collapsed variation will show the button again.
  • Auto expand - auto expands all collapsed variations in a study chapter or analysis tree the first time they are loaded

Commands

The commands feature is actually a combination of tools that register themselves with the cliCommands tool. This enables you to type / and then a command and execute various functions, besides the standard ones. Since commands are executed manually, they have no visible Preference to disable them.

Here is the list of available commands:

  • board - it will show/hide the board in Analysis mode (Analysis board and Studies). Use this with the Expanded move list and Hide left side options of the Move list options feature to have a move list filling the whole screen (good for presenting, for example).
  • trapvalue - provided the current move is in the Explorer database, this command will calculate the trap value of the current position. For more details on what that means, check out How To Calculate The Trappiest Chess Openings Using The Lichess API. 
  • copypgn ["fen"] ["separate"] ["tohere"] ["unicode"] - this will copy the moves in the analysis move list (Analysis board and Studies) to the clipboard as the moves reaching the current position and any branching moves from it.
    • if fen is specified, the PGNs will start from the current position with the FEN tag specified
      • same as Shift-click on the Copy PGN menu item 
    • if separate is specified, each variation will be saved as a separate linear PGN
      • same as Ctrl-click on the Copy PGN menu item
    • if tohere is specified, the PGN only copies moves to the current position
      • same as Alt-click on the Copy PGN menu item
    • if unicode is specified, the PGN will have unicode pieces instead of letters
      • note that results in a text that is meant for human readers, no longer a valid PGN
  • readgame [speed] [voice] [instrument] - this will read the game moves from the current position. It will follow only the first variation of every node, ignoring the others.
    • speed is an optional integer number, defaults to 100. Larger values makes the speech faster, lower makes it slower.
    • voice is an optional integer index number, defaults to 0. Changing it will change the voice which reads the moves, depending on your own browser setup.
    • instrument is an optional integer index number, defaults to 0. Changing it will enable a sound to be played in the background, based on the computer evaluation, if the game has been analyzed.
    • example: /readgame 80 2 1
    • this command also adds a feature to read a game in analysis if you add #readgame at the end of the URL (i.e. https://lichess.org/<game id>#readgame or https://lichess.org/<game id>/black#readgame or https://lichess.org/study/<study id>/<chapter id>#readgame etc.)
  • skipmove - this will generate 5 moves (if possible) to reach the same position, but with the other side to play.

Also save options for browser private/incognito mode

category: General - Advanced
values: yes/no - defaults to no

This was a user request to find a way to save the options outside the local browser cache, as he was playing the game from incognito/private browser mode. Since no other solution was better, this saves the options as notes in the first chess game you ever played on lichess. So be careful that this will alter those notes, although I am not aware of many people using the private notes feature.

Notes:

  • be aware that it will alter the private notes of the very first game you played on lichess
  • the extension will use whatever options you have in the browser local cache. Only if they are not there it will take them from the notes. That means that you can do weird stuff like enabling the feature, saving options, disabling it, changing the options and now you will have different settings in normal and incognito mode. So use with care.

Hide score tally crosstable

category: Play - Advanced
values: yes/no - defaults to no

Another user request was to not see the crosstable, the thing that shows how many games you played with the same opponent and what the results and games were. It was intimidating. This feature will blur it out, but allow you to click on it to unblur it. It's a silly feature that may not survive long because there is also the Zen mode lichess option which overlaps this behavior, but it's here for the moment.

and has 58 comments

  This extension adds a lot of functionalities to your Lichess web site. It has so many useful and powerful features! I am very proud of it. The extension is always going to be free, ad-free, donation links free, etc. Yet the only way for it to do what YOU want is feedback. Any feedback! Praise, curses, bug reports, feature requests, use stories, anything. The more you tell me, the more I can improve on this!

  I have also written a different page that will function as a user manual, with all the details on features, preferences and what they mean.

  If you are just interested in the list of features, in reversed chronological order, you might want to check out the history file.

  Join the LiChess Tools user team to get updates as they come, ask for features, give feedback, tell your stories!

  Help translate the extension in your language on Crowdin.

  Other stuff:

  • all features have been encapsulated in "tools" in the code
  • ideas for the future can be found as issues on GitHub, where you can also put feature requests and bug reports
  • the extension requires Chrome version 111 or higher or Firefox 130 or higher

And now back to what makes LiChess Tools great:

  LiChess Tools (ver. 2.3.164) adds the following features to lichess:

  • play ALL variations in Interactive lesson study chapters!
    • computer is going to play a random move (configurable probability), so you don't need to create a chapter for every small variation
  • PGN editor to merge, normalize, split, count, upload, download, copy PGNs.
  • merge multiple PGNs in analysis import
    • I merged 1000 PGNs with 25000 moves and it worked!
  • automatically open/hide/convert to menu or button the Friends box at page load
    • having the friends box as a menu/button item is really neat
  • sound alert when one of your friends starts playing a game
    • also reading the type of game, so you know if you even want to look at it
    • now there is an option to mute this for individual players in the enhanced friends list
  • ability to randomly play one of the next moves (with configurable probability in comments i.e. prc:66) with Ctrl-Right and go back with Ctrl-Left
  • play against the Opening Explorer (either masters, lichess players or a specific player) in Analysis
  • evaluation of Explorer moves, as well as telling you what move leads to gambits
  • Missed Timeline posts or comments to posts you follow notification
  • screen lock on mobiles while playing (scroll and zoom)
  • find interesting/brilliant moves and allowing cycling through interesting/good/brilliant moves just like with blunders
  • highlights for the last move of variations (special case for the ones that have no comment and do not end in checkmate) in the analysis/study board
    • you immediately see not only where a variation starts, but also where it ends
  • highlights for the transpositions to the current move in the analysis/study board
    • you won't ever have to worry that you are analyzing the exact same variation but in a different order
    • also you can now show all transposing positions in the PGN
  • new shortcut for playing the next best computer move from Space to Ctrl-Space
    • always annoyed me when I accidentally pressed the key
  • a custom chess engine level
    • if it is idle in a lower state, it runs until it gets to that level
    • this is also used as the required engine level by the study context menu option of commenting all last moves with a computer evaluation
  • custom chess engine options: never use cloud/tablebase, use engine in Practice mode
  • sticky Interactive lesson Preview mode
    • you can now play chapter after chapter without hassle
  • use keyboard shortcuts (i, m, b, Alt-i, Alt-m, Alt-b) for inaccuracies, mistakes and blunders in all variations
    • note that this is a native feature of lichess, but only in your game analyses and only the mainline moves
    • added g and Alt-g to cycle between "positive" moves (good, brilliant and interesting) 
  • show player country flags next to their names
    • if they have their country specified in the profile
    • now you will see flags everywhere. It might break some stuff, so let me know.
  • show the order of circles and arrows in a study/analysis.
    • this is great when you want to understand the order of moves/hints
    • option is off by default
  • a new menu item to open the last viewed TV game
  • show opening names in TV and mini games, as well as Analysis board and Studies
  • many TV options:
    • show history section in player TV (just like for category TV - the two latest games of the player)
    • friends and streamers section in the Current Games tab
    • link and bookmark the current TV game
  • quick button to switch to your player and back in personal opening explorer
  • copy to clipboard branch and continuations from a certain position in analysis/study
    • you can now just pick a variation, copy it in its own chapter, with just a few clicks
    • Shift/Ctrl/Alt change the way this item works
  • available languages: English and Romanian
    • ask for more! I will provide you with the English sentences and the context and you can tell me how it is in your language
  • the options for the extension are in the lichess Preferences section
    • complete integration. The extension popup has no functional role anymore
    • this also means that I will be able to port this to other browsers with minimal effort. Ask if you want this!
  • move options from transpositions to the current position
    • the Extended Interactive Lessons and the Ctrl-Arrow functionalities are also able to choose moves following from this list, as well as the variation arrows
  • automatically evaluate last moves in every variation and store it as a comment
    • the engine level for the evaluation is the same as the custom chess engine level in Preferences 
  • buttons in the study chapter edit form to quickly set the title to the content of the Event or of the White/Black PGN tags
  • set colors/styles to study comments
    • note that these will only be visible to people having the extension installed
  • study chapter navigation controls, including random chapter button (also with keyboard shortcut)
  • auto save and button to reload PGNs in Analysis mode (recover from accidental reloads) 
    • now it automatically copies the last PGN in the PGN box, but you have to manually import it by pressing the button
  • show all transpositions in the analysis/study move list
  • hide the score tally while playing
  • live friends page will update automatically and allow TV view, mute playing alerts and much more
  • global switch to enable/disable extension
  • ability to selectively remove artifacts (comments, shapes, glyphs and PGN tags) from the current study chapter
  • custom chat buttons at beginning and end of play
  • one button delete PGN tags
  • draw arrows and circles on mobile devices (analysis and in-game)
  • extra lines on the game analysis chart and local engine chart in analysis board
  • menu entry to go to last opened Study
  • study options: persist settings, create chapter after current, show chapter PGN as in Analysis
  • move list options: indented variations shows all variations as tree branches, not inline, expanded move list uses all the space available for the analysis move list and hide left side hides the left side of the analysis window for even more space. Open in new window lets you see the move list in another window that you can move to another monitor. You can have the computer evaluation toggle back on the right side.
  • bookmark study moves, which allows for collapse/expand variations, linking to position, highlight in the move list, getting the bookmark URL from a context menu and split the chapter from any bookmark.
  • Option to not see cloud values in computer evaluation
  • Wiki pages will now load in Analysis regardless of move order
  • Variation arrows for transpositions
  • Show pawn structure names in TV games, mini games, Analysis board and Studies
  • Click on Explorer total row to get a random move
  • Toggle between different Explorer lichess tab settings
  • Custom mini-game size
  • Play again from same position you entered Preview mode in
  • Use Stockfish 16 on Brave browser
  • Learn from your mistakes in study chapters
  • Pin studies and broadcasts to home page, hide spoilers in broadcasts
  • Community forum
  • Freeze board keyboard shortcut in Analysis/Study
  • Player lag chart next to player names during play
  • Link to download all studies of a user
  • Show profile chart time range dates in a label
  • Outside board coordinates, even in Analysis/Study, and bigger font.
  • Puzzle statistics in Profile
  • Move assistant shows the evaluation of your selected piece legal moves
  • Mirror button in Board Editor
  • More decimals in computer eval
  • auto unselect piece after a few seconds
  • Study flairs
  • Customize lobby page elements
  • Explorer resize
  • Custom sound options (just disable move sounds, keep the rest)
  • Back to current position in correspondence
  • Hide chat during play
  • Broadcast OBS support
  • Hide header shortcut
  • Next move behavior for variations (like Chessbase)
  • Autosave blog and/or button to manually save it and continue working
  • Active tab icon to see when a game move is made on inactive tabs
  • Keep screen active when watching TV
  • Themes to improve piece grab mechanism and visuals for both 2D and 3D boards
  • Fast interactive lesson moves
  • Paste images in chat/forum
  • No annoying chat warning about leaving Lichess
  • URL/image detection and unlimited text size in the team/study chat
  • one click moves in Analysis/Study
  • show common teams of you and your opponent
  • disable automatic collapsing of variations
  • video support in studies (so you can create courses)
  • external engines options
  • daily chess quote
  • copy puzzle PGN
  • game list filtering and selection
  • puzzle stats - chose your theme, replay failed puzzles, time interval slider
  • /commands! Type /help to get a list

  I couldn't wait to share it with you guys. I will be happy for any feedback, suggestions or help.

  I've started a series of use case blog posts, they might show you how to use the extension in real life:

Here are some screenshots, but they don't really tell you the story. You just have to try it.

Good luck using my extension. I am sure I am going to be tinkering with it a bit. Let me know of any problems you have with it.

Other ideas

For readability sake, I've removed all the old ideas from here and moved all of the new ones as GitHub issues. You can go there and add your own!

Q&A

Q: Can you publish your extension code on GitHub?
A: Yes, I did. I could. Probably I will be starting with version 2, which will be a rewrite of a version 1 that has been in use for a while and that people have given me feedback for. As much as I like sharing my code, I really don't want to have to deal with all the GitHub complications right now.

Q: I looked at your code and it sucks balls!
A: That's not a question. And I agree. But right now I am focusing on features, not quality control.

Q: How do we contact you with new ideas, bug reports and general roasting of your coding skills?
A: This is my personal blog and on the top-right of the page you can see a lot of links to various methods of direct communication with me. Comment here, use Discord, chat on Lichess with TotalNoob69, use the GitHub project. Any of these are perfectly acceptable.

Q: I am addicted to LiChess Tools and I am afraid later on you will fill it with ads, premium features and EULAs that allow you to remove my kidneys. Can you address my fear?
A: Like everything on this blog, it will always remain free. And not free as in "until someone else buys it" or free as in "watch videos and it's free" or free as in "I will fill your screen with junk", but completely utterly free. Like LiChess, I guess. Also, it doesn't connect to any external services or capture any user data. For now! Muhahahaha! Later on it might need some external services for extra features that you ask for, but I hope it won't have to.

Q: There are so many options that my brain hurts. And every time you add something new, my Lichess experience changes. I don't like this!
A: I am currently considering building a wizard that will adapt to your usage of Lichess and ask questions that will customize the extension for you. But in the meanwhile, on the bottom of the Preferences page there is a button that can turn every feature off. Then you can opt in to any feature you want.

Q: How long did it take you to write this?
A: Mostly a week. Following the 80/20 rule, now I have to work at least one more month to make it good. In the end it probably took two months to start and I am still tinkering, but I can only work on it when I get the time. This has been published since the 10th of May 2023 and I am still adding or fixing or changing things. For the cause!

Q: You should write a tutorial on how to use it. Could you make a video of it?
A: I am not a video person. I hope that this post can convey the basic ways in which to use the extension and that the extension itself can be used without the need of a tutorial. Let's work together to make this clear and easy to use for everyone instead. Also, there is now the user manual page. However, I am not adverse to someone who knows how making videos to make some about LiChess Tools. In fact, that would be absolutely amazing!

Q: Your Extended Interactive Lesson feature is all I had ever wanted from life! But when I am editing the study, I get the same interface as normal studies. Can you fix it?
A: Some parts of LiChess are easy to change, some not so much. Anything related to rendering is a mess to hook to. Additionally, I wouldn't want to have studies that can only be edited and used with my extension. There is a move context menu that allows setting the "explain why any other move is wrong" now. Also you can collapse the controls now, so they don't bother you at least.

Q: So how do I mark the good branches from the bad variations in Extended Interactive Lessons?
A: Any move that is not in the study will be bad. As for the branches that you want to explore specifically, use the annotations (Mistake, Blunder, Brilliant Move, etc) and comments. You can even explore the bad branches in Preview mode this way and learn why they would be bad.

Q: Can you add features to show me what moves to make while playing?
A: LiChess Tools is not a cheating tool. However I try to add as many tools as possible to help you analyze your games after you've played them.

Q: But can you add some features that don't involve cheating for the games that I am playing/watching?
A: Most of the features of LiChess Tools are analysis oriented because analysis is much better exposed than the game code. Because there are a lot of private variables that are not made accessible, it's difficult to selectively change parts of the game interface and any features would have to brutally copy paste and replace some legitimate code bits. I am afraid that until that changes on LiChess, I will not touch that part, mostly because that means I would have to keep score on what they change on the web site and update my extension accordingly. Also, there are some guidelines that expect one to not change the playing interface at all. It makes sense, as any edge LiChess Tools might provide to a player could be construed as cheating.

Q: How about changing the way LiChess looks?
A: I am not a good visual designer, nor do I do a lot of work on web frontend. There are some extensions that are doing that (like Prettier Lichess, which I used myself), and perhaps you should ask those people for help instead. Also, I am avoiding as much as possible changes to the visual elements of the website specifically because it might interfere with some such extension or custom CSS tool. BTW, if you are working on something like that and find LiChess Tools is interfering with your stuff, let me know. We can figure things out. In v2.0.14 a new Themes tool has been added. I can publish CSS themes this way, but I don't intend to maintain them myself. If you want to see your theme there, contact me. 

Q: OK, you're my new hero. How can I help?
A: Contact me and let's talk. I despise doing anything UI design related, as evidenced by this blog and the extension popup, so maybe you can help there. You can help with algorithms to analyze games better or find useful information from the tidbits that Lichess exposes. Anything, really, just as long as it's fun for you.

Q: Yeah, but I can't code. How can I help?
A: Help me by making this extension known. I don't want "marketing", just spread the word. Let people know and if they like it, they will use it. Can't use it if they don't know about it, though, and I am always afraid people think I am spamming them when I try to advertise my work. Make this famous, is it too much to ask?

Q: I use LiChess in my own language and the new features are jarring in English
A: The only languages I personally support are English and Romanian. Other languages are supported via Crowdin. Unfortunately, Crowdin has a limit of words that can be translated for free, so that limits the number of languages available to about 10 more. I configured the ones that are used most or have the most potential in terms of active speakers on the Internet. You can help translate there, because for each of those languages I started with the default machine translation which I am pretty sure sucks.

Q: Chrome sucks! Microsoft sold out! I hate Firefox! Can you make this work for my favorite browser?
A: Short answer: no. Long answer: I want to help people, so the more the merrier, but I also don't have a lot of resources to maintain code on a browser I don't use. Safari is a mess and extensions on it require to have a tool that only works on Macs and they ask you for money. Firefox has less than 5% of the market and refuses to implement the feature that makes LiChess Tools work. Firefox is supported now. Opera already supports Chrome extensions. To be honest, it is not reasonable for me to bother with anything but what is supported now. So long answer is also no :)

A: That's not my bug, it comes from LiChess. They have bugs, too.
Q: How could you possibly have answered before I asked the question?

Q: Did you actually think people were going to read this far down?
A: No.

Q: I told about this to all my friends, I came with feedback and constructive criticism and it feels like you ignored me. What gives?
A: For sure I want to take everything into consideration and act on requests as fast as possible, but it might be that I am caught up with something else. I thoroughly intend to give you and the extension as much attention as possible, so maybe make sure I got your message, first.

Hope it helps!

and has 0 comments

C# 3.0 introduced Object Initializer Syntax which was a game changer for code that created new objects or new collections. Here is a contrived example:

var obj = new ComplexObject
{
    // object initializer
    AnItem = new Item("text1"),
    AnotherItem = new Item("text2"),
    // collection initializer
    RestOfItems = new List<Item>
    {
        new Item("text3"),
        new Item("text4"),
        new Item("text5")
    },
    // indexer initializer
    [0]=new Item("text6"),
    [1]=new Item("text7")
};

Before this syntax was available, the same code would have looked like this:

var obj = new ComplexObject();
obj.AnItem = new Item("text1");
obj.AnotherItem = new Item("text2");
obj.RestOfItems = new List<Item>();
obj.RestOfItems.Add(new Item("text3"));
obj.RestOfItems.Add(new Item("text4"));
obj.RestOfItems.Add(new Item("text5"));
obj[0] = new Item("text6");
obj[2] = new Item("text7");

It's not like the number of lines has changed, but both the writability and readability of the code increase with the new syntax. At least that's why I think. However, outside these very simple scenarios, the feature feels like it's encumbering us or that it is missing something. Imagine you want to only add items to a list based on some condition. You might get a code like this:

var list = new List<Item>
{
    new Item("text1")
};
if (condition) list.Add(new Item("text2"));

We use the initializer for one item, but not for the other. We might as well use Add for both items, then, or use some cumbersome syntax that hurts more than it helps:

var list = new[]
{
    new Item("text1"),
    condition?new Item("text2"):null
}
.Where(i => i != null)
.ToList();

It's such an ugly syntax that Visual Studio doesn't know how to indent it properly. What to do? Software patterns to the rescue! 

Seriously now, people who know me know that I scoff at the general concept of software patterns, but the patterns themselves are useful and in this case, even the conceptual framework that I often deride is useful here. Because we are trying to initialize an object or a collection, which means we are attempting to build it. So why not use a Builder pattern? Here are two versions of the same code, one with extension methods (which can be used everywhere, but might pollute the member list for common objects) and another with an actual builder object created specifically for our purposes (which may simplify usage):

// extension methods
var list = new List<Item>()
    .Adding(new Item("text1"))
    .ConditionalAdding(condition, new Item("text2"));
...
public static class ItemListExtensions
{
    public static List<T> Adding<T>(this List<T> list, T item)
    {
        list.Add(item);
        return list;
    }
    public static List<T> ConditionalAdding<T>(this List<T> list, bool condition, T item)
    {
        if (condition)
        {
            list.Add(item);
        }
        return list;
    }
}

// builder object
var list = new ItemListBuilder()
    .Adding("text1")
    .ConditionalAdding(condition, "text2")
    .Build();
...
public class ItemListBuilder
{
    private readonly List<Item> list;

    public ItemListBuilder()
    {
        list = new List<Item>();
    }

    public ItemListBuilder Adding(string text)
    {
        list.Add(new Item(text));
        return this;
    }

    public ItemListBuilder ConditionalAdding(bool condition, string text)
    {
        if (condition)
        {
            list.Add(new Item(text));
        }
        return this;
    }

    public List<Item> Build()
    {
        return list.ToList();
    }
}

Of course, for just a simple collection with some conditions this might feel like overkill, but try to compare the two versions of the code: the one that uses initializer syntax and then the Add method and the one that declares what it wants to do, step by step. Also note that in the case of the builder object I've taken the liberty of creating methods that only take string parameters then build the list of Item, thus simplifying the syntax and clarifying intent.

I had this situation where I had to map an object to another object by copying some properties into collections and values of some type to other types and so on. The original code was building the output using a top-down approach:

public Output BuildOutput(Input input) {
  var output=new Output();
  BuildFirstPart(output, input);
  BuildSecondPart(output, input);
  ...
  return output;
}

public BuildFirstPart(Output output, Input input) {
  var firstSection = BuildFirstSection(input);
  output.FirstPart=new List<Part> {
    new Part(firstSection)
  };
  if (condition) {
    var secondSection=BuildSeconfSection(input);
    output.FirstPart.Add(new Part(secondSection));
  }
}

And so on and so on. I believe that in this case a fluent design makes the code a lot more readable:

var output = new Output {
  FirstPart = new List<Part>()
    .Adding(BuildFirstSection(input))
    .ConditionalAdding(condition, BuildSecondSection(input),
  SecondPart = ...
};

The "build section" methods would also be inlined and replaced with fluent design methods. In this way the structure of "the output" is clearly shown in a method that declares what it will build and populates the various members of the Output class with simple calculations, the only other methods that the builder needs. A human will understand at a glance what thing it will build, see its structure as a tree of code and be able to go to individual methods to see or change the specific calculation that provides a value.

The point of my post is that sometimes the very out-of-the-box features that help us a lot most of the time end up complicating and obfuscating our code in specific situations. If the code starts to smell, to become unreadable, to make you feel bad for writing it, then stop, think of a better solution, then implement it so that it is the best version for your particular case. Use tools when they are useful and discard them when other solutions might prove more effective.

and has 1 comment

Intro

  Some of the most visited posts on this blog relate to dependency injection in .NET. As you may know, dependency injection has been baked in in ASP.Net almost since the beginning, but it culminated with the MVC framework and the .Net Core rewrite. Dependency injection has been separated into packages from where it can be used everywhere. However, probably because they thought it was such a core concept or maybe because it is code that came along since the days of UnityContainer, the entire mechanism is sealed, internalized and without any hooks on which to add custom code. Which, in my view, is crazy, since dependency injection serves, amongst other things, the purpose of one point of change for class instantiations.

  Now, to be fair, I am not an expert in the design patterns used in dependency injection in the .NET code. There might be some weird way in which you can extend the code that I am unaware of. In that case, please illuminate me. But as far as I went in the code, this is the simplest way I found to insert my own hook into the resolution process. If you just want the code, skip to the end.

Using DI

  First of all, a recap on how to use dependency injection (from scratch) in a console application:

// you need the nuget packages Microsoft.Extensions.DependencyInjection 
// and Microsoft.Extensions.DependencyInjection.Abstractions
using Microsoft.Extensions.DependencyInjection;
...

// create a service collection
var services = new ServiceCollection();
// add the mappings between interface and implementation
services.AddSingleton<ITest, Test>();
// build the provider
var provider = services.BuildServiceProvider();

// get the instance of a service
var test = provider.GetService<ITest>();

  Note that this is a very simplified scenario. For more details, please check Creating a console app with Dependency Injection in .NET Core.

Recommended pattern for DI

  Second of all, a recap of the recommended way of using dependency injection (both from Microsoft and myself) which is... constructor injection. It serves two purposes:

  1. It declares all the dependencies of an object in the constructor. You can rest assured that all you would ever need for that thing to work is there.
  2. When the constructor starts to fill a page you get a strong hint that your class may be doing too many things and you should split it up.

  But then again, there is the "Learn the rules. Master the rules. Break the rules" concept. I've familiarized myself with it before writing this post so that now I can safely break the second part and not master anything before I break stuff. I am talking now about property injection, which is generally (for good reason) frowned upon, but which one may want to use in scenarios adjacent to the functionality of the class, like logging. One of the things that always bothered me is having to declare a logger in every constructor ever, even if in itself a logger does nothing to the functionality of the class.

  So I've had this idea that I would use constructor dependency injection EVERYWHERE, except logging. I would create an ILogger<T> property which would be automatically injected with the correct implementation at resolution time. Only there is a problem: Microsoft's dependency injection does not support property injection or resolution hooks (as far as I could find). So I thought of a solution.

How does it work?

  Third of all, a small recap on how ServiceProvider really works.

  When one does services.BuildServiceProvider() they actually call an extension method that does new ServiceProvider(services, someServiceProviderOptions). Only that constructor is internal, so you can't use it yourself. Then, inside the provider class, the GetService method is using a ConcurrentDictionary of service accessors to get your service. In case the service accessor is not there, the method from the field _createServiceAccessor is going to be used. So my solution: replace the field value with a wrapper that will also execute our own code.

The solution

  Before I show you the code, mind that this applies to .NET 7.0. I guess it will work in most .NET Core versions, but they could change the internal field name or functionality in which case this might break.

  Finally, here is the code:

public static class ServiceProviderExtensions
{
    /// <summary>
    /// Adds a custom handler to be executed after service provider resolves a service
    /// </summary>
    /// <param name="provider">The service provider</param>
    /// <param name="handler">An action receiving the service provider, 
    /// the registered type of the service 
    /// and the actual instance of the service</param>
    /// <returns>the same ServiceProvider</returns>
    public static ServiceProvider AddCustomResolveHandler(this ServiceProvider provider,
                 Action<IServiceProvider, Type, object> handler)
    {
        var field = typeof(ServiceProvider).GetField("_createServiceAccessor",
                        BindingFlags.Instance | BindingFlags.NonPublic);
        var accessor = (Delegate)field.GetValue(provider);
        var newAccessor = (Type type) =>
        {
            Func<object, object> newFunc = (object scope) =>
            {
                var resolver = (Delegate)accessor.DynamicInvoke(new[] { type });
                var resolved = resolver.DynamicInvoke(new[] { scope });
                handler(provider, type, resolved);
                return resolved;
            };
            return newFunc;
        };
        field.SetValue(provider, newAccessor);
        return provider;
    }
}

  As you can see, we take the original accessor delegate and we replace it with a version that runs our own handler immediately after the service has been instantiated.

Populating a Logger property

  And we can use it like this to do property injection now:

static void Main(string[] args)
{
    var services = new ServiceCollection();
    services.AddSingleton<ITest, Test>();
    var provider = services.BuildServiceProvider();
    provider.AddCustomResolveHandler(PopulateLogger);

    var test = (Test)provider.GetService<ITest>();
    Assert.IsNotNull(test.Logger);
}

private static void PopulateLogger(IServiceProvider provider, 
                                    Type type, object service)
{
    if (service is null) return;
    var propInfo = service.GetType().GetProperty("Logger",
                    BindingFlags.Instance|BindingFlags.Public);
    if (propInfo is null) return;
    var expectedType = typeof(ILogger<>).MakeGenericType(service.GetType());
    if (propInfo.PropertyType != expectedType) return;
    var logger = provider.GetService(expectedType);
    propInfo.SetValue(service, logger);
}

  See how I've added the PopulateLogger handler in which I am looking for a property like 

public ILogger<Test> Logger { get; private set; }

  (where the generic type of ILogger is the same as the class) and populate it.

Populating any decorated property

  Of course, this is kind of ugly. If you want to enable property injection, why not use an attribute that makes your intention clear and requires less reflection? Fine. Let's do it like this:

// Add handler
provider.AddCustomResolveHandler(InjectProperties);
...

// the handler populates all properties that are decorated with [Inject]
private static void InjectProperties(IServiceProvider provider, Type type, object service)
{
    if (service is null) return;
    var propInfos = service.GetType()
        .GetProperties(BindingFlags.Instance | BindingFlags.Public)
        .Where(p => p.GetCustomAttribute<InjectAttribute>() != null)
        .ToList();
    foreach (var propInfo in propInfos)
    {
        var instance = provider.GetService(propInfo.PropertyType);
        propInfo.SetValue(service, instance);
    }
}
...

// the attribute class
[AttributeUsage(AttributeTargets.Property, AllowMultiple = false, Inherited = true)]
public class InjectAttribute : Attribute {}

Conclusion

I have demonstrated how to add a custom handler to be executed after any service instance is resolved by the default Microsoft ServiceProvider class, which in turn enables property injection, one point of change to all classes, etc. I once wrote code to wrap any class into a proxy that would trace all property and method calls with their parameters automatically. You can plug that in with the code above, if you so choose.

Be warned that this solution is using reflection to change the functionality of the .NET 7.0 ServiceProvider class and, if the code there changes for some reason, you might need to adapt it to the latest functionality.

If you know of a more elegant way of doing this, please let me know.

Hope it helps!

Bonus

But what about people who really, really, really hate reflection and don't want to use it? What about situations where you have a dependency injection framework running for you, but you have no access to the service provider builder code? Isn't there any solution?

Yes. And No. (sorry, couldn't help myself)

The issue is that ServiceProvider, ServiceCollection and all that jazz are pretty closed up. There is no solution I know of that solved this issue. However... there is one particular point in the dependency injection setup which can be hijacked and that is... the adding of the service descriptors themselves!

You see, when you do ServiceCollection.AddSingleton<Something,Something>, what gets called is yet another extension method, the ServiceCollection itself is nothing but a list of ServiceDescriptor. The Add* extensions methods come from ServiceCollectionServiceExtensions class, which contains a lot of methods that all defer to just three different effects:

  • adding a ServiceDescriptor on a type (so associating an type with a concrete type) with a specific lifetime (transient, scoped or singleton)
  • adding a ServiceDescriptor on an instance (so associating a type with a specific instance of a class), by default singleton
  • adding a ServiceDescriptor on a factory method (so associating a type with a constructor method)

If you think about it, the first two can be translated into the third. In order to instantiate a type using a service provider you do ActivatorUtilities.CreateInstance(provider, type) and a factory method that returns a specific instance of a class is trivial.

So, the solution: just copy paste the contents of ServiceCollectionServiceExtensions and make all of the methods end up in the Add method using a service factory method descriptor. Now instead of using the extensions from Microsoft, you use your class, with the same effect. Next step: replace the provider factory method with a wrapper that also executes stuff.

Since this is a bonus, I let you implement everything except the Add method, which I will provide here:

// original code
private static IServiceCollection Add(
    IServiceCollection collection,
    Type serviceType,
    Func<IServiceProvider, object> implementationFactory,
    ServiceLifetime lifetime)
{
    var descriptor = new ServiceDescriptor(serviceType, implementationFactory, lifetime);
    collection.Add(descriptor);
    return collection;
}

//updated code
private static IServiceCollection Add(
    IServiceCollection collection,
    Type serviceType,
    Func<IServiceProvider, object> implementationFactory,
    ServiceLifetime lifetime)
{
    Func<IServiceProvider, object> factory = (sp)=> {
        var instance = implementationFactory(sp);
        // no stack overflow, please
        if (instance is IDependencyResolver) return instance;
        // look for a registered instance of IDependencyResolver (our own interface)
        var resolver=sp.GetService<IDependencyResolver>();
        // intercept the resolution and replace it with our own 
        return resolver?.Resolve(sp, serviceType, instance) ?? instance;
    };
    var descriptor = new ServiceDescriptor(serviceType, factory, lifetime);
    collection.Add(descriptor);
    return collection;
}

All you have to do is (create the interface and then) register your own implementation of IDependencyResolver and do whatever you want to do in the Resolve method, including the logger instantiation, the inject attribute handling or the wrapping of objects, as above. All without reflection.

The kick here is that you have to make sure you don't use the default Add* methods when you register your services, or this won't work. 

There you have it, bonus content not found on dev.to ;)