View source with raw comments or as raw
    1:- module(language_server, [language_server/0, language_server/1, stop_language_server/1]).    2
    3% To generate docs:
    4% - Open SWI Prolog
    5% - consult("/.../swiplserver/swiplserver/language_server.pl")
    6% - doc_save("/.../swiplserver/swiplserver/language_server.pl", [doc_root("/.../swiplserver/docs/language_server")]).
    7
    8/*  Prolog Language Server
    9    Author:        Eric Zinda
   10    E-mail:        ericz@inductorsoftware.com
   11    WWW:           http://www.inductorsoftware.com
   12    Copyright (c)  2021, Eric Zinda
   13    All rights reserved.
   14
   15        Redistribution and use in source and binary forms, with or without
   16    modification, are permitted provided that the following conditions
   17    are met:
   18
   19    1. Redistributions of source code must retain the above copyright
   20       notice, this list of conditions and the following disclaimer.
   21
   22    2. Redistributions in binary form must reproduce the above copyright
   23       notice, this list of conditions and the following disclaimer in
   24       the documentation and/or other materials provided with the
   25       distribution.
   26
   27    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
   28    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
   29    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
   30    FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
   31    COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
   32    INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
   33    BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
   34    LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
   35    CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
   36    LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
   37    ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
   38    POSSIBILITY OF SUCH DAMAGE.
   39*/
 language_server(+Options:list) is semidet
Starts a Prolog language server using Options. The server is normally started automatically by a library built for a particular programming language such as the swiplserver Python library, but starting manually can be useful when debugging Prolog code in some scenarios. See the documentation on "Standalone Mode" for more information.

Once started, the server listens for TCP/IP or Unix Domain Socket connections and authenticates them using the password provided before processing any messages. The messages processed by the server are described below.

For debugging, the server outputs traces using the debug/3 predicate so that the server operation can be observed by using the debug/1 predicate. Run the following commands to see them:

Options

Options is a list containing any combination of the following options. When used in the Prolog top level (i.e. in Standalone Mode), these are specified as normal Prolog options like this:

language_server([unix_domain_socket(Socket), password('a password')])

When using "Embedded Mode" they are passed using the same name but as normal command line arguments like this:

swipl --quiet -g language_server -t halt -- --write_connection_values=true --password="a password" --create_unix_domain_socket=true

Note the use of quotes around values that could confuse command line processing like spaces (e.g. "a password") and that unix_domain_socket(Variable) is written as --create_unix_domain_socket=true on the command line. See below for more information.

port(?Port)
The TCP/IP port to bind to on localhost. This option is ignored if the unix_domain_socket/1 option is set. Port is either a legal TCP/IP port number (integer) or a variable term like Port. If it is a variable, it causes the system to select a free port and unify the variable with the selected port as in tcp_bind/2. If the option write_connection_values(true) is set, the selected port is output to STDOUT followed by \n on startup to allow the client language library to retrieve it in "Embedded Mode".
unix_domain_socket(?Unix_Domain_Socket_Path_And_File)
If set, Unix Domain Sockets will be used as the way to communicate with the server. Unix_Domain_Socket_Path_And_File specifies the fully qualified path and filename to use for the socket.

To have one generated instead (recommended), pass Unix_Domain_Socket_Path_And_File as a variable when calling from the Prolog top level and the variable will be unified with a created filename. If launching in "Embedded Mode", instead pass --create_unix_domain_socket=true since there isn't a way to specify variables from the command line. When generating the file, a temporary directory will be created using tmp_file/2 and a socket file will be created within that directory following the below requirements. If the directory and file are unable to be created for some reason, language_server/1 fails.

Regardless of whether the file is specified or generated, if the option write_connection_values(true) is set, the fully qualified path to the generated file is output to STDOUT followed by \n on startup to allow the client language library to retrieve it.

Specifying a file to use should follow the same guidelines as the generated file:

  • If the file exists when the server is launched, it will be deleted.
  • The Prolog process will attempt to create and, if Prolog exits cleanly, delete this file (and directory if it was created) when the server closes. This means the directory from a specified file must have the appropriate permissions to allow the Prolog process to do so.
  • For security reasons, the filename should not be predictable and the directory it is contained in should have permissions set so that files created are only accessible to the current user.
  • The path must be below 92 bytes long (including null terminator) to be portable according to the Linux documentation.
password(?Password)
The password required for a connection. If not specified (recommended), the server will generate one as a Prolog string type since Prolog atoms are globally visible (be sure not to convert to an atom for this reason). If Password is a variable it will be unified with the created password. Regardless of whether the password is specified or generated, if the option write_connection_values(true) is set, the password is output to STDOUT followed by \n on startup to allow the client language library to retrieve it. This is the recommended way to integrate the server with a language as it avoids including the password as source code. This option is only included so that a known password can be supplied for when the server is running in Standalone Mode.
query_timeout(+Seconds)
Sets the default time in seconds that a query is allowed to run before it is cancelled. This can be overridden on a query by query basis. If not set, the default is no timeout (-1).
pending_connections(+Count)
Sets the number of pending connections allowed for the server as in tcp_listen/2. If not provided, the default is 5.
run_server_on_thread(+Run_Server_On_Thread)
Determines whether language_server/1 runs in the background on its own thread or blocks until the server shuts down. Must be missing or set to true when running in "Embedded Mode" so that the SWI Prolog process can exit properly. If not set, the default is true.
server_thread(?Server_Thread)
Specifies or retrieves the name of the thread the server will run on if run_server_on_thread(true). Passing in an atom for Server_Thread will only set the server thread name if run_server_on_thread(true). If Server_Thread is a variable, it is unified with a generated name.
write_connection_values(+Write_Connection_Values)
Determines whether the server writes the port (or generated Unix Domain Socket) and password to STDOUT as it initializes. Used by language libraries to retrieve this information for connecting. If not set, the default is false.
write_output_to_file(+File)
Redirects STDOUT and STDERR to the file path specified. Useful for debugging the server when it is being used in "Embedded Mode". If using multiple servers in one SWI Prolog instance, only set this on the first one. Each time it is set the output will be redirected.

Language Server Messages

The messages the server responds to are described below. A few things are true for all of them:

Language Server Message Format

Every language server message is a single valid Prolog term. Those that run queries have an argument which represents the query as a single term. To run several goals at once use (goal1, goal2, ...) as the goal term.

The format of sent and received messages is identical (\n stands for the ASCII newline character which is a single byte):

<stringByteLength>.\n<stringBytes>.\n.

For example, to send hello as a message you would send this:

7.\nhello.\n

To send a message to the server, send a message using the message format above to the localhost port or Unix Domain Socket that the server is listening on. For example, to run the synchronous goal atom(a), send the following message:

18.\nrun(atom(a), -1).\n<end of stream>

You will receive the response below on the receive stream of the same connection you sent on. Note that the answer is in JSON format. If a message takes longer than 2 seconds, there will be "heartbeat" characters (".") at the beginning of the response message, approximately 1 every 2 seconds. So, if the query takes 6 seconds for some reason, there will be three "." characters first:

...12\ntrue([[]]).\n

Language Server Messages Reference

The full list of language server messages are described below:

run(Goal, Timeout)
Runs Goal on the connection's designated query thread. Stops accepting new commands until the query is finished and it has responded with the results. If a previous query is still in progress, waits until the previous query finishes (discarding that query's results) before beginning the new query.

Timeout is in seconds and indicates a timeout for generating all results for the query. Sending a variable (e.g. _) will use the default timeout passed to the initial language_server/1 predicate and -1 means no timeout.

While it is waiting for the query to complete, sends a "." character not in message format, just as a single character, once every two seconds to proactively ensure that the client is alive. Those should be read and discarded by the client.

If a communication failure happens (during a heartbeat or otherwise), the connection is terminated, the query is aborted and (if running in "Embedded Mode") the SWI Prolog process shuts down.

When completed, sends a response message using the normal message format indicating the result.

Response:

true([Answer1, Answer2, ... ])The goal succeeded at least once. The response always includes all answers as if run with findall() (see run_async/3 below to get individual results back iteratively). Each Answer is a list of the assignments of free variables in the answer. If there are no free variables, Answer is an empty list.
falseThe goal failed.
exception(time_limit_exceeded)The query timed out.
exception(Exception)An arbitrary exception was not caught while running the goal.
exception(connection_failed)The query thread unexpectedly exited. The server will no longer be listening after this exception.
run_async(Goal, Timeout, Find_All)
Starts a Prolog query specified by Goal on the connection's designated query thread. Answers to the query, including exceptions, are retrieved afterwards by sending the async_result message (described below). The query can be cancelled by sending the cancel_async message. If a previous query is still in progress, waits until that query finishes (discarding that query's results) before responding.

Timeout is in seconds and indicates a timeout for generating all results for the query. Sending a variable (e.g. _) will use the default timeout passed to the initial language_server/1 predicate and -1 means no timeout.

If the socket closes before a response is sent, the connection is terminated, the query is aborted and (if running in "Embedded Mode") the SWI Prolog process shuts down.

If it needs to wait for the previous query to complete, it will send heartbeat messages (see "Language Server Message Format") while it waits. After it responds, however, it does not send more heartbeats. This is so that it can begin accepting new commands immediately after responding so the client.

Find_All == true means generate one response to an async_result message with all of the answers to the query (as in the run message above). Find_All == false generates a single response to an async_result message per answer.

Response:

true([[]])The goal was successfully parsed.
exception(Exception)An error occurred parsing the goal.
exception(connection_failed)The goal thread unexpectedly shut down. The server will no longer be listening after this exception.
cancel_async
Attempt to cancel a query started by the run_async message in a way that allows further queries to be run on this Prolog thread afterwards.

If there is a goal running, injects a throw(cancel_goal) into the executing goal to attempt to stop the goal's execution. Begins accepting new commands immediately after responding. Does not inject abort/0 because this would kill the connection's designated thread and the system is designed to maintain thread local data for the client. This does mean it is a "best effort" cancel since the exception can be caught.

cancel_async is guaranteed to either respond with an exception (if there is no query or pending results from the last query), or safely attempt to stop the last executed query even if it has already finished.

To guarantee that a query is cancelled, send close and close the socket.

It is not necessary to determine the outcome of cancel_async after sending it and receiving a response. Further queries can be immediately run. They will start after the current query stops.

However, if you do need to determine the outcome or determine when the query stops, send async_result. Using Timeout = 0 is recommended since the query might have caught the exception or still be running. Sending async_result will find out the "natural" result of the goal's execution. The "natural" result depends on the particulars of what the code actually did. The response could be:

exception(cancel_goal)The query was running and did not catch the exception. I.e. the goal was successfully cancelled.
exception(time_limit_exceeded)The query timed out before getting cancelled.
exception(Exception)They query hits another exception before it has a chance to be cancelled.
A valid answerThe query finished before being cancelled.

Note that you will need to continue sending async_result until you receive an exception(Exception) message if you want to be sure the query is finished (see documentation for async_result).

Response:

true([[]])There is a query running or there are pending results for the last query.
exception(no_query)There is no query or pending results from a query to cancel.
exception(connection_failed)The connection has been unexpectedly shut down. The server will no longer be listening after this exception.
async_result(Timeout)
Get results from a query that was started via a run_async message. Used to get results for all cases: if the query terminates normally, is cancelled by sending a cancel_async message, or times out.

Each response to an async_result message responds with one result and, when there are no more results, responds with exception(no_more_results) or whatever exception stopped the query. Receiving any exception response except exception(result_not_available) means there are no more results. If run_async was run with Find_All == false, multiple async_result messages may be required before receiving the final exception.

Waits Timeout seconds for a result. Timeout == -1 or sending a variable for Timeout indicates no timeout. If the timeout is exceeded and no results are ready, sends exception(result_not_available).

Some examples:

If the query succeeds with N answers...async_result messages 1 to N will receive each answer, in order, and async_result message N+1 will receive exception(no_more_results)
If the query fails (i.e. has no answers)...async_result message 1 will receive false and async_result message 2 will receive exception(no_more_results)
If the query times out after one answer...async_result message 1 will receive the first answer and async_result message 2 will receive exception(time_limit_exceeded)
If the query is cancelled after it had a chance to get 3 answers...async_result messages 1 to 3 will receive each answer, in order, and async_result message 4 will receive exception(cancel_goal)
If the query throws an exception before returning any results...async_result message 1 will receive exception(Exception)

Note that, after sending cancel_async, calling async_result will return the "natural" result of the goal's execution. The "natural" result depends on the particulars of what the code actually did since this is multi-threaded and there are race conditions. This is described more below in the response section and above in cancel_async.

Response:

true([Answer1, Answer2, ... ])The next answer from the query is a successful answer. Whether there are more than one Answer in the response depends on the findall setting. Each Answer is a list of the assignments of free variables in the answer. If there are no free variables, Answer is an empty list.
falseThe query failed with no answers.
exception(no_query)There is no query in progress.
exception(result_not_available)There is a running query and no results were available in Timeout seconds.
exception(no_more_results)There are no more answers and no other exception occurred.
exception(cancel_goal)The next answer is an exception caused by cancel_async. Indicates no more answers.
exception(time_limit_exceeded)The query timed out generating the next answer (possibly in a race condition before getting cancelled). Indicates no more answers.
exception(Exception)The next answer is an arbitrary exception. This can happen after cancel_async if the cancel_async exception is caught or the code hits another exception first. Indicates no more answers.
exception(connection_failed)The goal thread unexpectedly exited. The server will no longer be listening after this exception.
close
Closes a connection cleanly, indicating that the subsequent socket close is not a connection failure. Thus it doesn't shutdown the server in "Embedded Mode". The response must be processed by the client before closing the socket or it will be interpreted as a connection failure.

Any asynchronous query that is still running will be halted by using abort/0 in the connection's query thread.

Response: true([[]])

quit
Stops the server and ends the SWI Prolog process. This allows client language libraries to ask for an orderly shutdown of the Prolog process.

Response: true([[]])

*/

  250:- use_module(library(socket)).  251:- use_module(library(http/json)).  252:- use_module(library(http/json_convert)).  253:- use_module(library(option)).  254:- use_module(library(term_to_json)).  255% One for every language server running
  256:- dynamic(language_server_thread/3).  257
  258% One for every active connection
  259:- dynamic(language_server_worker_threads/3).  260:- dynamic(language_server_socket/5).  261
  262% Indicates that a query is in progress on the goal thread or hasn't had its results drained
  263% Deleted once the last result from the queue has been drained
  264% Only deleted by the communication thread to avoid race conditions
  265:- dynamic(query_in_progress/1).  266
  267% Indicates to the communication thread that we are in a place
  268% that can be cancelled
  269:- dynamic(safe_to_cancel/1).  270
  271
  272% Password is carefully constructed to be a string (not an atom) so that it is not
  273% globally visible
  274% Add ".\n" to the password since it will be added by the message when received
  275language_server(Options) :-
  276    Encoding = utf8,
  277    option(pending_connections(Connection_Count), Options, 5),
  278    option(query_timeout(Query_Timeout), Options, -1),
  279    option(port(Port), Options, _),
  280    option(run_server_on_thread(Run_Server_On_Thread), Options, true),
  281    option(exit_main_on_failure(Exit_Main_On_Failure), Options, false),
  282    option(write_connection_values(Write_Connection_Values), Options, false),
  283    option(unix_domain_socket(Unix_Domain_Socket_Path_And_File), Options, _),
  284    (   (   memberchk(unix_domain_socket(_), Options),
  285            var(Unix_Domain_Socket_Path_And_File)
  286        )
  287    ->  unix_domain_socket_path(Unix_Domain_Socket_Path, Unix_Domain_Socket_Path_And_File)
  288    ;   true
  289    ),
  290    option(server_thread(Server_Thread_ID), Options, _),
  291    (   var(Server_Thread_ID)
  292    ->  gensym(language_server, Server_Thread_ID)
  293    ;   true
  294    ),
  295    option(password(Password), Options, _),
  296    (   var(Password)
  297    ->  (   uuid(UUID, [format(integer)]),
  298            format(string(Password), '~d', [UUID])
  299        )
  300    ;   true
  301    ),
  302    string_concat(Password, '.\n', Final_Password),
  303    bind_socket(Server_Thread_ID, Unix_Domain_Socket_Path_And_File, Port, Socket, Client_Address),
  304    send_client_startup_data(Write_Connection_Values, user_output, Unix_Domain_Socket_Path_And_File, Client_Address, Password),
  305    option(write_output_to_file(File), Options, _),
  306    (   var(File)
  307    ->  true
  308    ;   write_output_to_file(File)
  309    ),
  310    Server_Goal = (
  311                    catch(server_thread(Server_Thread_ID, Socket, Client_Address, Final_Password, Connection_Count, Encoding, Query_Timeout, Exit_Main_On_Failure), error(E1, E2), true),
  312                    debug(language_server(protocol), "Stopped server on thread: ~w due to exception: ~w", [Server_Thread_ID, error(E1, E2)])
  313                 ),
  314    start_server_thread(Run_Server_On_Thread, Server_Thread_ID, Server_Goal, Unix_Domain_Socket_Path, Unix_Domain_Socket_Path_And_File).
 language_server is semidet
Main entry point for running the Language Server in "Embedded Mode" and designed to be called from the command line. Embedded Mode is used when launching the Language Server as an embedded part of another language (e.g. Python). Calling language_server/0 from Prolog interactively is not recommended as it depends on Prolog exiting to stop the server, instead use language_server/1 for interactive use.

To launch embedded mode:

swipl --quiet -g language_server -t halt -- --write_connection_values=true

This will start SWI Prolog and invoke the language_server/0 predicate and exit the process when that predicate stops. Any command line arguments after the standalone -- will be passed as Options. These are the same Options that language_server/1 accepts and are passed to it directly. Some options are expressed differently due to command line limitations, see language_server/1 Options for more information.

Any Option values that causes issues during command line parsing (such as spaces) should be passed with "" like this:

swipl --quiet -g language_server -t halt -- --write_connection_values=true --password="HGJ SOWLWW WNDSJD"
  335% Turn off int signal when running in embedded mode so the client language
  336% debugger signal doesn't put Prolog into debug mode
  337% run_server_on_thread must be missing or true (the default) so we can exit
  338% properly
  339% create_unix_domain_socket=true/false is only used as a command line argument
  340% since it doesn't seem possible to pass create_unix_domain_socket=_ on the command line
  341% and have it interpreted as a variable.
  342language_server :-
  343    current_prolog_flag(os_argv, Argv),
  344    argv_options(Argv, _Args, Options),
  345    append(Options, [exit_main_on_failure(true)], Options1),
  346    select_option(create_unix_domain_socket(Create_Unix_Domain_Socket), Options1, Options2, false),
  347    (   Create_Unix_Domain_Socket
  348    ->  append(Options2, [unix_domain_socket(_)], FinalOptions)
  349    ;   FinalOptions = Options2
  350    ),
  351    option(run_server_on_thread(Run_Server_On_Thread), FinalOptions, true),
  352    (   Run_Server_On_Thread
  353    ->  true
  354    ;   throw(domain_error(cannot_be_set_in_embedded_mode, run_server_on_thread))
  355    ),
  356    language_server(FinalOptions),
  357    on_signal(int, _, quit),
  358    thread_get_message(quit_language_server).
  359
  360
  361quit(_) :-
  362    thread_send_message(main, quit_language_server).
 stop_language_server(+Server_Thread_ID:atom) is det
If Server_Thread_ID is a variable, stops all language servers and associated threads. If Server_Thread_ID is an atom, then only the server with that Server_Thread_ID is stopped. Server_Thread_ID can be provided or retrieved using Options in language_server/1.

Always succeeds.

  371% tcp_close_socket(Socket) will shut down the server thread cleanly so the socket is released and can be used again in the same session
  372% Closes down any pending connections using abort even if there were no matching server threads since the server thread could have died.
  373% At this point only threads associated with live connections (or potentially a goal thread that hasn't detected its missing communication thread)
  374% should be left so seeing abort warning messages in the console seems OK
  375stop_language_server(Server_Thread_ID) :-
  376    % First shut down any matching servers to stop new connections
  377    forall(retract(language_server_thread(Server_Thread_ID, _, Socket)),
  378        (
  379            debug(language_server(protocol), "Found server: ~w", [Server_Thread_ID]),
  380            catch(tcp_close_socket(Socket), Socket_Exception, true),
  381            abortSilentExit(Server_Thread_ID, Server_Thread_Exception),
  382            debug(language_server(protocol), "Stopped server thread: ~w, socket_close_exception(~w), stop_thread_exception(~w)", [Server_Thread_ID, Socket_Exception, Server_Thread_Exception])
  383        )),
  384    forall(retract(language_server_worker_threads(Server_Thread_ID, Communication_Thread_ID, Goal_Thread_ID)),
  385        (
  386            abortSilentExit(Communication_Thread_ID, CommunicationException),
  387            debug(language_server(protocol), "Stopped server: ~w communication thread: ~w, exception(~w)", [Server_Thread_ID, Communication_Thread_ID, CommunicationException]),
  388            abortSilentExit(Goal_Thread_ID, Goal_Exception),
  389            debug(language_server(protocol), "Stopped server: ~w goal thread: ~w, exception(~w)", [Server_Thread_ID, Goal_Thread_ID, Goal_Exception])
  390        )).
  391
  392
  393start_server_thread(Run_Server_On_Thread, Server_Thread_ID, Server_Goal, Unix_Domain_Socket_Path, Unix_Domain_Socket_Path_And_File) :-
  394    (   Run_Server_On_Thread
  395    ->  (   thread_create(Server_Goal, _, [ alias(Server_Thread_ID),
  396                                            at_exit((delete_unix_domain_socket_file(Unix_Domain_Socket_Path, Unix_Domain_Socket_Path_And_File),
  397                                                     detach_if_expected(Server_Thread_ID)
  398                                                    ))
  399                                          ]),
  400            debug(language_server(protocol), "Started server on thread: ~w", [Server_Thread_ID])
  401        )
  402    ;   (   Server_Goal,
  403            delete_unix_domain_socket_file(Unix_Domain_Socket_Path, Unix_Domain_Socket_Path_And_File),
  404            debug(language_server(protocol), "Halting.", [])
  405        )
  406    ).
  407
  408
  409% Unix domain sockets create a file that needs to be cleaned up
  410% If language_server generated it, there is also a directory that needs to be cleaned up
  411%   that will only contain that file
  412delete_unix_domain_socket_file(Unix_Domain_Socket_Path, Unix_Domain_Socket_Path_And_File) :-
  413    (   nonvar(Unix_Domain_Socket_Path)
  414    ->  catch(delete_directory_and_contents(Unix_Domain_Socket_Path), error(_, _), true)
  415    ;   (   nonvar(Unix_Domain_Socket_Path_And_File)
  416        ->  catch(delete_file(Unix_Domain_Socket_Path_And_File), error(_, _), true)
  417        ;   true
  418        )
  419    ).
  420
  421:- if(current_predicate(unix_domain_socket/1)).  422    optional_unix_domain_socket(Socket) :-
  423        unix_domain_socket(Socket).
  424:- else.  425    optional_unix_domain_socket(_).
  426:- endif.  427
  428% Always bind only to localhost for security reasons
  429% Delete the socket file in case it is already around so that the same name can be reused
  430bind_socket(Server_Thread_ID, Unix_Domain_Socket_Path_And_File, Port, Socket, Client_Address) :-
  431    (   nonvar(Unix_Domain_Socket_Path_And_File)
  432    ->  debug(language_server(protocol), "Using Unix domain socket name: ~w", [Unix_Domain_Socket_Path_And_File]),
  433        optional_unix_domain_socket(Socket),
  434        catch(delete_file(Unix_Domain_Socket_Path_And_File), error(_, _), true),
  435        tcp_bind(Socket, Unix_Domain_Socket_Path_And_File),
  436        Client_Address = Unix_Domain_Socket_Path_And_File
  437    ;   (   tcp_socket(Socket),
  438            tcp_setopt(Socket, reuseaddr),
  439            tcp_bind(Socket, '127.0.0.1':Port),
  440            debug(language_server(protocol), "Using TCP/IP port: ~w", ['127.0.0.1':Port]),
  441            Client_Address = Port
  442        )
  443    ),
  444    assert(language_server_thread(Server_Thread_ID, Unix_Domain_Socket_Path_And_File, Socket)).
  445
  446% Communicates the used port and password to the client via STDOUT so the client
  447% language library can use them to connect
  448send_client_startup_data(Write_Connection_Values, Stream, Unix_Domain_Socket_Path_And_File, Port, Password) :-
  449    (   Write_Connection_Values
  450    ->  (   (  var(Unix_Domain_Socket_Path_And_File)
  451            ->  format(Stream, "~d\n", [Port])
  452            ;   format(Stream, "~w\n", [Unix_Domain_Socket_Path_And_File])
  453            ),
  454            format(Stream, "~w\n", [Password]),
  455            flush_output(Stream)
  456        )
  457    ;   true
  458    ).
  459
  460
  461% Server thread worker predicate
  462% Listen for connections and create a connection for each in its own communication thread
  463% Uses tail recursion to ensure the stack doesn't grow
  464server_thread(Server_Thread_ID, Socket, Address, Password, Connection_Count, Encoding, Query_Timeout, Exit_Main_On_Failure) :-
  465    debug(language_server(protocol), "Listening on address: ~w", [Address]),
  466    tcp_listen(Socket, Connection_Count),
  467    tcp_open_socket(Socket, AcceptFd, _),
  468    create_connection(Server_Thread_ID, AcceptFd, Password, Encoding, Query_Timeout, Exit_Main_On_Failure),
  469    server_thread(Server_Thread_ID, Socket, Address, Password, Connection_Count, Encoding, Query_Timeout, Exit_Main_On_Failure).
  470
  471
  472% Wait for the next connection and create communication and goal threads to support it
  473% Create known IDs for the threads so we can pass them along before the threads are created
  474% First create the goal thread to avoid a race condition where the communication
  475% thread tries to queue a goal before it is created
  476create_connection(Server_Thread_ID, AcceptFd, Password, Encoding, Query_Timeout, Exit_Main_On_Failure) :-
  477    debug(language_server(protocol), "Waiting for client connection...", []),
  478    tcp_accept(AcceptFd, Socket, _Peer),
  479    debug(language_server(protocol), "Client connected", []),
  480    gensym('conn', Connection_Base),
  481    atomic_list_concat([Server_Thread_ID, "_", Connection_Base, '_comm'], Thread_Alias),
  482    atomic_list_concat([Server_Thread_ID, "_", Connection_Base, '_goal'], Goal_Alias),
  483    mutex_create(Goal_Alias, [alias(Goal_Alias)]),
  484    assert(language_server_worker_threads(Server_Thread_ID, Thread_Alias, Goal_Alias)),
  485    thread_create(goal_thread(Thread_Alias),
  486        _,
  487        [alias(Goal_Alias), at_exit(detach_if_expected(Goal_Alias))]),
  488    thread_create(communication_thread(Password, Socket, Encoding, Server_Thread_ID, Goal_Alias, Query_Timeout, Exit_Main_On_Failure),
  489        _,
  490        [alias(Thread_Alias), at_exit(detach_if_expected(Thread_Alias))]).
  491
  492
  493% The worker predicate for the Goal thread.
  494% Looks for a message from the connection thread, processes it, then recurses.
  495%
  496% Goals always run in the same thread in case the user is setting thread local information.
  497% For each answer to the user's query (including an exception), the goal thread will queue a message
  498% to the communication thread of the form result(Answer, Find_All), where Find_All == true if the user wants all answers at once
  499% Tail recurse to avoid growing the stack
  500goal_thread(Respond_To_Thread_ID) :-
  501    thread_self(Self_ID),
  502    throw_if_testing(Self_ID),
  503    thread_get_message(Self_ID, goal(Goal, Binding_List, Query_Timeout, Find_All)),
  504    debug(language_server(query), "Received Findall = ~w, Query_Timeout = ~w, binding list: ~w, goal: ~w", [Find_All, Query_Timeout, Binding_List, Goal]),
  505    (   Find_All
  506    ->  One_Answer_Goal = findall(Binding_List, @(user:Goal, user), Answers)
  507    ;
  508        One_Answer_Goal = ( @(user:Goal, user),
  509                            Answers = [Binding_List],
  510                            send_next_result(Respond_To_Thread_ID, Answers, _, Find_All)
  511                          )
  512    ),
  513    All_Answers_Goal = run_cancellable_goal(Self_ID, findall(Answers, One_Answer_Goal, [Find_All_Answers | _])),
  514    (   Query_Timeout == -1
  515    ->  catch(All_Answers_Goal, Top_Exception, true)
  516    ;   catch(call_with_time_limit(Query_Timeout, All_Answers_Goal), Top_Exception, true)
  517    ),
  518    (
  519        var(Top_Exception)
  520    ->  (
  521            Find_All
  522        ->
  523            send_next_result(Respond_To_Thread_ID, Find_All_Answers, _, Find_All)
  524        ;
  525            send_next_result(Respond_To_Thread_ID, [], no_more_results, Find_All)
  526        )
  527    ;
  528        send_next_result(Respond_To_Thread_ID, [], Top_Exception, true)
  529    ),
  530    goal_thread(Respond_To_Thread_ID).
  531
  532
  533% Used only for testing unhandled exceptions outside of the "safe zone"
  534throw_if_testing(Self_ID) :-
  535    (   thread_peek_message(Self_ID, testThrow(Test_Exception))
  536    ->  (   debug(language_server(query), "TESTING: Throwing test exception: ~w", [Test_Exception]),
  537            throw(Test_Exception)
  538        )
  539    ;   true
  540    ).
  541
  542
  543% run_cancellable_goal handles the communication
  544% to ensure the cancel exception from the communication thread
  545% is injected at a place we are prepared to handle in the goal_thread
  546% Before the goal is run, sets a fact to indicate we are in the "safe to cancel"
  547% zone for the communication thread.
  548% Then it doesn't exit this "safe to cancel" zone if the
  549% communication thread is about to cancel
  550run_cancellable_goal(Mutex_ID, Goal) :-
  551    thread_self(Self_ID),
  552    setup_call_cleanup(
  553        assert(safe_to_cancel(Self_ID), Assertion),
  554        Goal,
  555        with_mutex(Mutex_ID, erase(Assertion))
  556    ).
  557
  558
  559% Worker predicate for the communication thread.
  560% Processes messages and sends goals to the goal thread.
  561% Continues processing messages until communication_thread_listen() throws or ends with true/false
  562%
  563% Catches all exceptions from communication_thread_listen so that it can do an orderly shutdown of the goal
  564%   thread if there is a communication failure.
  565%
  566% True means user explicitly called close or there was an exception
  567%   only exit the main thread if there was an exception and we are supposed to Exit_Main_On_Failure
  568%   otherwise just exit the session
  569communication_thread(Password, Socket, Encoding, Server_Thread_ID, Goal_Thread_ID, Query_Timeout, Exit_Main_On_Failure) :-
  570    thread_self(Self_ID),
  571    (   (
  572            catch(communication_thread_listen(Password, Socket, Encoding, Server_Thread_ID, Goal_Thread_ID, Query_Timeout), error(Serve_Exception1, Serve_Exception2), true),
  573            debug(language_server(protocol), "Session finished. Communication thread exception: ~w", [error(Serve_Exception1, Serve_Exception2)]),
  574            abortSilentExit(Goal_Thread_ID, _),
  575            retractall(language_server_worker_threads(Server_Thread_ID, Self_ID, Goal_Thread_ID))
  576        )
  577    ->  Halt = (nonvar(Serve_Exception1), Exit_Main_On_Failure)
  578    ;   Halt = true
  579    ),
  580    (   Halt
  581    ->  (   debug(language_server(protocol), "Ending session and halting Prolog server due to thread ~w: exception(~w)", [Self_ID, error(Serve_Exception1, Serve_Exception2)]),
  582            quit(_)
  583        )
  584    ;   (   debug(language_server(protocol), "Ending session ~w", [Self_ID]),
  585            catch(tcp_close_socket(Socket), error(_, _), true)
  586        )
  587    ).
  588
  589
  590% Open socket and begin processing the streams for a connection using the Encoding if the password matches
  591% true: session ended
  592% exception: communication failure or an internal failure (like a thread threw or shutdown unexpectedly)
  593% false: halt
  594communication_thread_listen(Password, Socket, Encoding, Server_Thread_ID, Goal_Thread_ID, Query_Timeout) :-
  595    tcp_open_socket(Socket, Read_Stream, Write_Stream),
  596    thread_self(Communication_Thread_ID),
  597    assert(language_server_socket(Server_Thread_ID, Communication_Thread_ID, Socket, Read_Stream, Write_Stream)),
  598    set_stream(Read_Stream, encoding(Encoding)),
  599    set_stream(Write_Stream, encoding(Encoding)),
  600    read_message(Read_Stream, Sent_Password),
  601    (   Password == Sent_Password
  602    ->  (   debug(language_server(protocol), "Password matched.", []),
  603            thread_self(Self_ID),
  604            reply(Write_Stream, true([[threads(Self_ID, Goal_Thread_ID)]]))
  605        )
  606    ;   (   debug(language_server(protocol), "Password mismatch, failing. ~w", [Sent_Password]),
  607            reply_error(Write_Stream, password_mismatch),
  608            throw(password_mismatch)
  609        )
  610    ),
  611    process_language_server_messages(Read_Stream, Write_Stream, Goal_Thread_ID, Query_Timeout),
  612    debug(language_server(protocol), "Session finished.", []).
  613
  614
  615% process_language_server_messages implements the main interface to the language server.
  616% Continuously reads a language server message from Read_Stream and writes a response to Write_Stream,
  617% until the connection fails or a `quit` or `close` message is sent.
  618%
  619% Read_Stream and Write_Stream can be any valid stream using any encoding.
  620%
  621% Goal_Thread_ID must be the threadID of a thread started on the goal_thread predicate
  622%
  623% uses tail recursion to ensure the stack doesn't grow
  624%
  625% true: indicates we should terminate the session (clean termination)
  626% false: indicates we should exit the process if running in embedded mode
  627% exception: indicates we should terminate the session (communication failure termination) or
  628%    thread was asked to halt
  629process_language_server_messages(Read_Stream, Write_Stream, Goal_Thread_ID, Query_Timeout) :-
  630    process_language_server_message(Read_Stream, Write_Stream, Goal_Thread_ID, Query_Timeout, Command),
  631    (   Command == close
  632    ->  (   debug(language_server(protocol), "Command: close. Client closed the connection cleanly.", []),
  633            true
  634        )
  635    ;   (   Command == quit
  636        ->  (   debug(language_server(protocol), "Command: quit.", []),
  637                false
  638            )
  639        ;
  640            process_language_server_messages(Read_Stream, Write_Stream, Goal_Thread_ID, Query_Timeout)
  641        )
  642    ).
  643
  644% process_language_server_message manages the protocol for the connection: receive message, parse it, process it.
  645% - Reads a single message from Read_Stream.
  646% - Processes it and issues a response on Write_Stream.
  647% - The message will be unified with Command to allow the caller to handle it.
  648%
  649% Read_Stream and Write_Stream can be any valid stream using any encoding.
  650%
  651% True if the message understood. A response will always be sent.
  652% False if the message was malformed.
  653% Exceptions will be thrown by the underlying stream if there are communication failures writing to Write_Stream or the thread was asked to exit.
  654%
  655% state_* predicates manage the state transitions of the protocol
  656% They only bubble up exceptions if there is a communication failure
  657%
  658% state_process_command will never return false
  659% since errors should be sent to the client
  660% It can throw if there are communication failures, though.
  661process_language_server_message(Read_Stream, Write_Stream, Goal_Thread_ID, Query_Timeout, Command) :-
  662    debug(language_server(protocol), "Waiting for next message ...", []),
  663    (   state_receive_raw_message(Read_Stream, Message_String)
  664    ->  (   state_parse_command(Write_Stream, Message_String, Command, Binding_List)
  665        ->  state_process_command(Write_Stream, Goal_Thread_ID, Query_Timeout, Command, Binding_List)
  666        ;   true
  667        )
  668    ;   false
  669    ).
  670
  671
  672% state_receive_raw_message: receive a raw message, which is simply a string
  673%   true: valid message received
  674%   false: invalid message format
  675%   exception: communication failure OR thread asked to exit
  676state_receive_raw_message(Read, Command_String) :-
  677    read_message(Read, Command_String),
  678    debug(language_server(protocol), "Valid message: ~w", [Command_String]).
  679
  680
  681% state_parse_command: attempt to parse the message string into a valid command
  682%
  683% Use read_term_from_atom instead of read_term(stream) so that we don't hang
  684% indefinitely if the caller didn't properly finish the term
  685% parse in the context of module 'user' to properly bind operators, do term expansion, etc
  686%
  687%   true: command could be parsed
  688%   false: command cannot be parsed.  An error is sent to the client in this case
  689%   exception: communication failure on sending a reply
  690state_parse_command(Write_Stream, Command_String, Parsed_Command, Binding_List) :-
  691    (   catch(read_term_from_atom(Command_String, Parsed_Command, [variable_names(Binding_List), module(user)]), Parse_Exception, true)
  692    ->  (   var(Parse_Exception)
  693        ->  debug(language_server(protocol), "Parse Success: ~w", [Parsed_Command])
  694        ;   (   reply_error(Write_Stream, Parse_Exception),
  695                fail
  696            )
  697        )
  698    ;   (   reply_error(Write_Stream, error(couldNotParseCommand, _)),
  699            fail
  700        )
  701    ).
  702
  703
  704% state_process_command(): execute the requested Command
  705%
  706% First wait until we have removed all results from any previous query.
  707% If query_in_progress(Goal_Thread_ID) exists then there is at least one
  708% more result to drain, by definition. Because the predicate is
  709% deleted by get_next_result in the communication thread when the last result is drained
  710%
  711%   true: if the command itself succeeded, failed or threw an exception.
  712%         In that case, the outcome is sent to the client
  713%   exception: only communication or thread failures are allowed to bubble up
  714% See language_server(Options) documentation
  715state_process_command(Stream, Goal_Thread_ID, Query_Timeout, run(Goal, Timeout), Binding_List) :-
  716    !,
  717    debug(language_server(protocol), "Command: run/1. Timeout: ~w", [Timeout]),
  718    repeat_until_false((
  719            query_in_progress(Goal_Thread_ID),
  720            debug(language_server(protocol), "Draining unretrieved result for ~w", [Goal_Thread_ID]),
  721            heartbeat_until_result(Goal_Thread_ID, Stream, Unused_Answer),
  722            debug(language_server(protocol), "Drained result for ~w", [Goal_Thread_ID]),
  723            debug(language_server(query), "    Discarded answer: ~w", [Unused_Answer])
  724        )),
  725    debug(language_server(protocol), "All previous results drained", []),
  726    send_goal_to_thread(Stream, Goal_Thread_ID, Query_Timeout, Timeout, Goal, Binding_List, true),
  727    heartbeat_until_result(Goal_Thread_ID, Stream, Answers),
  728    reply_with_result(Goal_Thread_ID, Stream, Answers).
  729
  730
  731% See language_server(Options) documentation for documentation
  732% See notes in run(Goal, Timeout) re: draining previous query
  733state_process_command(Stream, Goal_Thread_ID, Query_Timeout, run_async(Goal, Timeout, Find_All), Binding_List) :-
  734    !,
  735    debug(language_server(protocol), "Command: run_async/1.", []),
  736    debug(language_server(query),  "   Goal: ~w", [Goal]),
  737    repeat_until_false((
  738            query_in_progress(Goal_Thread_ID),
  739            debug(language_server(protocol), "Draining unretrieved result for ~w", [Goal_Thread_ID]),
  740            heartbeat_until_result(Goal_Thread_ID, Stream, Unused_Answer),
  741            debug(language_server(protocol), "Drained result for ~w", [Goal_Thread_ID]),
  742            debug(language_server(query), "    Discarded answer: ~w", [Unused_Answer])
  743            )),
  744    debug(language_server(protocol), "All previous results drained", []),
  745    send_goal_to_thread(Stream, Goal_Thread_ID, Query_Timeout, Timeout, Goal, Binding_List, Find_All),
  746    reply(Stream, true([[]])).
  747
  748
  749% See language_server(Options) documentation for documentation
  750state_process_command(Stream, Goal_Thread_ID, _, async_result(Timeout), _) :-
  751    !,
  752    debug(language_server(protocol), "Command: async_result, timeout: ~w.", [Timeout]),
  753    (   once((var(Timeout) ; Timeout == -1))
  754    ->  Options = []
  755    ;   Options = [timeout(Timeout)]
  756    ),
  757    (   query_in_progress(Goal_Thread_ID)
  758    ->  (   (   debug(language_server(protocol), "Pending query results exist for ~w", [Goal_Thread_ID]),
  759                get_next_result(Goal_Thread_ID, Stream, Options, Result)
  760            )
  761        ->  reply_with_result(Goal_Thread_ID, Stream, Result)
  762        ;   reply_error(Stream, result_not_available)
  763        )
  764   ;    (   debug(language_server(protocol), "No pending query results for ~w", [Goal_Thread_ID]),
  765            reply_error(Stream, no_query)
  766        )
  767   ).
  768
  769
  770% See language_server(Options) documentation for documentation
  771% To ensure the goal thread is in a place it is safe to cancel,
  772% we lock a mutex first that the goal thread checks before exiting
  773% the "safe to cancel" zone.
  774% It is not in the safe zone: it either finished
  775% or was never running.
  776state_process_command(Stream, Goal_Thread_ID, _, cancel_async, _) :-
  777    !,
  778    debug(language_server(protocol), "Command: cancel_async/0.", []),
  779    with_mutex(Goal_Thread_ID, (
  780        (   safe_to_cancel(Goal_Thread_ID)
  781        ->  (   thread_signal(Goal_Thread_ID, throw(cancel_goal)),
  782                reply(Stream, true([[]]))
  783            )
  784        ;   (   query_in_progress(Goal_Thread_ID)
  785            ->  (   debug(language_server(protocol), "Pending query results exist for ~w", [Goal_Thread_ID]),
  786                    reply(Stream, true([[]]))
  787                )
  788            ;   (   debug(language_server(protocol), "No pending query results for ~w", [Goal_Thread_ID]),
  789                    reply_error(Stream, no_query)
  790                )
  791            )
  792        )
  793    )).
  794
  795
  796% Used for testing how the system behaves when the goal thread is killed unexpectedly
  797% Needs to run a bogus command `run(true, -1)` to
  798% get the goal thread to process the exception
  799state_process_command(Stream, Goal_Thread_ID, Query_Timeout, testThrowGoalThread(Test_Exception), Binding_List) :-
  800    !,
  801    debug(language_server(protocol), "TESTING: requested goal thread unhandled exception", []),
  802    thread_send_message(Goal_Thread_ID, testThrow(Test_Exception)),
  803    state_process_command(Stream, Goal_Thread_ID, Query_Timeout, run(true, -1), Binding_List).
  804
  805
  806state_process_command(Stream, _, _, close, _) :-
  807    !,
  808    reply(Stream, true([[]])).
  809
  810
  811state_process_command(Stream, _, _, quit, _) :-
  812    !,
  813    reply(Stream, true([[]])).
  814
  815
  816%  Send an exception if the command is not known
  817state_process_command(Stream, _, _, Command, _) :-
  818    debug(language_server(protocol), "Unknown command ~w", [Command]),
  819    reply_error(Stream, unknownCommand).
  820
  821
  822% Wait for a result (and put in Answers) from the goal thread, but send a heartbeat message
  823% every so often until it arrives to detect if the socket is broken.
  824% Throws if If the heartbeat failed which will
  825% and then shutdown the communication thread
  826% Tail recurse to not grow the stack
  827heartbeat_until_result(Goal_Thread_ID, Stream, Answers) :-
  828    (   get_next_result(Goal_Thread_ID, Stream, [timeout(2)], Answers)
  829    ->  debug(language_server(query), "Received answer from goal thread: ~w", [Answers])
  830    ;   (   debug(language_server(protocol), "heartbeat...", []),
  831            write_heartbeat(Stream),
  832            heartbeat_until_result(Goal_Thread_ID, Stream, Answers)
  833        )
  834    ).
  835
  836
  837% True if write succeeded, otherwise throws as that
  838% indicates that heartbeat failed because the other
  839% end of the pipe terminated
  840write_heartbeat(Stream) :-
  841    put_char(Stream, '.'),
  842    flush_output(Stream).
  843
  844
  845% Send a goal to the goal thread in its queue
  846%
  847% Remember that we are now running a query using assert.
  848%   This will be retracted once all the answers have been drained.
  849%
  850% If Goal_Thread_ID died, thread_send_message throws and, if we don't respond,
  851%   the client could hang so catch and give them a good message before propagating
  852%   the exception
  853send_goal_to_thread(Stream, Goal_Thread_ID, Default_Timeout, Timeout, Goal, Binding_List, Find_All) :-
  854    (   var(Timeout)
  855    ->  Timeout = Default_Timeout
  856    ;   true
  857    ),
  858    (   var(Binding_List)
  859    ->  Binding_List = []
  860    ;   true
  861    ),
  862    debug(language_server(query),  "Sending to goal thread with timeout = ~w: ~w", [Timeout, Goal]),
  863    assert(query_in_progress(Goal_Thread_ID)),
  864    catch(thread_send_message(Goal_Thread_ID, goal(Goal, Binding_List, Timeout, Find_All)), Send_Message_Exception, true),
  865    (   var(Send_Message_Exception)
  866    ->  true
  867    ;   (   reply_error(Stream, connection_failed),
  868            throw(Send_Message_Exception)
  869        )
  870    ).
  871
  872
  873% Send a result from the goal thread to the communication thread in its queue
  874send_next_result(Respond_To_Thread_ID, Answer, Exception_In_Goal, Find_All) :-
  875    (   var(Exception_In_Goal)
  876    ->  (   (   debug(language_server(query), "Sending result of goal to communication thread, Result: ~w", [Answer]),
  877                Answer == []
  878            )
  879        ->  thread_send_message(Respond_To_Thread_ID, result(false, Find_All))
  880        ;   thread_send_message(Respond_To_Thread_ID, result(true(Answer), Find_All))
  881        )
  882    ;   (   debug(language_server(query), "Sending result of goal to communication thread, Exception: ~w", [Exception_In_Goal]),
  883            thread_send_message(Respond_To_Thread_ID, result(error(Exception_In_Goal), Find_All))
  884        )
  885    ).
  886
  887
  888% Gets the next result from the goal thread in the communication thread queue,
  889% and retracts query_in_progress/1 when the last result has been sent.
  890% Find_All == true only returns one message, so delete query_in_progress
  891% No matter what it is
  892% \+ Find_All: There may be more than one result. The first one we hit with any exception
  893% (note that no_more_results is also returned as an exception) means we are done
  894get_next_result(Goal_Thread_ID, Stream, Options, Answers) :-
  895    (   thread_property(Goal_Thread_ID, status(running))
  896    ->  true
  897    ;   (   reply_error(Stream, connection_failed),
  898            throw(connection_failed)
  899        )
  900    ),
  901    thread_self(Self_ID),
  902    thread_get_message(Self_ID, result(Answers, Find_All), Options),
  903    (   Find_All
  904    ->  (   debug(language_server(protocol), "Query completed and answers drained for findall ~w", [Goal_Thread_ID]),
  905            retractall(query_in_progress(Goal_Thread_ID))
  906        )
  907    ;   (   Answers = error(_)
  908        ->  (   debug(language_server(protocol), "Query completed and answers drained for non-findall ~w", [Goal_Thread_ID]),
  909                retractall(query_in_progress(Goal_Thread_ID))
  910            )
  911        ;   true
  912        )
  913    ).
  914
  915
  916% reply_with_result predicates are used to consistently return
  917% answers for a query from either run() or run_async()
  918reply_with_result(_, Stream, error(Error)) :-
  919    !,
  920    reply_error(Stream, Error).
  921reply_with_result(_, Stream, Result) :-
  922    !,
  923    reply(Stream, Result).
  924
  925
  926% Reply with a normal term
  927% Convert term to an actual JSON string
  928reply(Stream, Term) :-
  929    debug(language_server(query), "Responding with Term: ~w", [Term]),
  930    term_to_json_string(Term, Json_String),
  931    write_message(Stream, Json_String).
  932
  933
  934% Special handling for exceptions since they can have parts that are not
  935% "serializable". Ensures they they are always returned in an exception/1 term
  936reply_error(Stream, Error_Term) :-
  937    (   error(Error_Value, _) = Error_Term
  938    ->  Response = exception(Error_Value)
  939    ;   (   atom(Error_Term)
  940        ->
  941            Response = exception(Error_Term)
  942        ;   (   compound_name_arity(Error_Term, Name, _),
  943                Response = exception(Name)
  944            )
  945        )
  946    ),
  947    reply(Stream, Response).
  948
  949
  950% Send and receive messages are simply strings preceded by their length + ".\n"
  951% i.e. "<stringlength>.\n<string>"
  952% The desired encoding must be set on the Stream before calling this predicate
  953
  954
  955% Writes the next message.
  956% Throws if there is an unexpected exception
  957write_message(Stream, String) :-
  958    write_string_length(Stream, String),
  959    write(Stream, String),
  960    flush_output(Stream).
  961
  962
  963% Reads the next message.
  964% Throws if there is an unexpected exception or thread has been requested to quit
  965% the length passed must match the actual number of bytes in the stream
  966% in whatever encoding is being used
  967read_message(Stream, String) :-
  968    read_string_length(Stream, Length),
  969    read_string(Stream, Length, String).
  970
  971
  972% Terminate with '.\n' so we know that's the end of the count
  973write_string_length(Stream, String) :-
  974    stream_property(Stream, encoding(Encoding)),
  975    string_encoding_length(String, Encoding, Length),
  976    format(Stream, "~d.\n", [Length]).
  977
  978
  979% Note: read_term requires ".\n" after the length
  980% ... but does not consume the "\n"
  981read_string_length(Stream, Length) :-
  982    read_term(Stream, Length, []),
  983    get_char(Stream, _).
  984
  985
  986% converts a string to Codes using Encoding
  987string_encoding_length(String, Encoding, Length) :-
  988    setup_call_cleanup(
  989        open_null_stream(Out),
  990        (   set_stream(Out, encoding(Encoding)),
  991            write(Out, String),
  992            byte_count(Out, Length)
  993        ),
  994        close(Out)).
  995
  996
  997% Convert Prolog Term to a Prolog JSON term
  998% Add a final \n so that using netcat to debug works well
  999term_to_json_string(Term, Json_String) :-
 1000    term_to_json(Term, Json),
 1001    with_output_to(string(Json_String),
 1002        (   current_output(Stream),
 1003            json_write(Stream, Json),
 1004            put(Stream, '\n')
 1005        )).
 1006
 1007
 1008% Execute the goal as once() without binding any variables
 1009% and keep executing it until it returns false (or throws)
 1010repeat_until_false(Goal) :-
 1011    (\+ (\+ Goal)), !, repeat_until_false(Goal).
 1012repeat_until_false(_).
 1013
 1014
 1015% Used to kill a thread in an "expected" way so it doesn't leave around traces in thread_property/2 afterwards
 1016%
 1017% If the thread is alive OR it was already aborted (expected cases) then attempt to join
 1018%   the thread so that no warnings are sent to the console. Other cases leave the thread for debugging.
 1019% There are some fringe cases (like calling external code)
 1020%   where the call might not return for a long time.  Do a timeout for those cases.
 1021abortSilentExit(Thread_ID, Exception) :-
 1022    catch(thread_signal(Thread_ID, abort), error(Exception, _), true),
 1023    debug(language_server(protocol), "Attempting to abort thread: ~w. thread_signal_exception: ~w", [Thread_ID, Exception]).
 1024% Workaround SWI Prolog bug: https://github.com/SWI-Prolog/swipl-devel/issues/852 by not joining
 1025% The workaround just stops joining the aborted thread, so an inert record will be left if thread_property/2 is called.
 1026%    ,
 1027%    (   once((var(Exception) ; catch(thread_property(Thread_ID, status(exception('$aborted'))), error(_, _), true)))
 1028%    ->  (   catch(call_with_time_limit(4, thread_join(Thread_ID)), error(JoinException1, JoinException2), true),
 1029%            debug(language_server(protocol), "thread_join attempted because thread: ~w exit was expected, exception: ~w", [Thread_ID, error(JoinException1, JoinException2)])
 1030%        )
 1031%    ;   true
 1032%    ).
 1033
 1034
 1035% Detach a thread that exits with true or false so that it doesn't leave around a record in thread_property/2 afterwards
 1036% Don't detach a thread if it exits because of an exception so we can debug using thread_property/2 afterwards
 1037%
 1038% However, `abort` is an expected exception but detaching a thread that aborts will leave an unwanted
 1039% thread_property/2 record *and* print a message to the console. To work around this,
 1040% the goal thread is always aborted by the communication thread using abortSilentExit.
 1041detach_if_expected(Thread_ID) :-
 1042    thread_property(Thread_ID, status(Status)),
 1043    debug(language_server(protocol), "Thread ~w exited with status ~w", [Thread_ID, Status]),
 1044    (   once((Status = true ; Status = false))
 1045    ->  (   debug(language_server(protocol), "Expected thread status, detaching thread ~w", [Thread_ID]),
 1046            thread_detach(Thread_ID)
 1047        )
 1048    ;   true
 1049    ).
 1050
 1051
 1052write_output_to_file(File) :-
 1053    debug(language_server(protocol), "Writing all STDOUT and STDERR to file:~w", [File]),
 1054    open(File, write, Stream, [buffer(false)]),
 1055    set_prolog_IO(user_input, Stream, Stream).
 1056
 1057
 1058% Creates a Unix Domain Socket file in a secured directory.
 1059% Throws if the directory or file cannot be created in /tmp for any reason
 1060% Requirements for this file are:
 1061%    - The Prolog process will attempt to create and, if Prolog exits cleanly,
 1062%           delete this file when the server closes.  This means the directory
 1063%           must have the appropriate permissions to allow the Prolog process
 1064%           to do so.
 1065%    - For security reasons, the filename should not be predictable and the
 1066%           directory it is contained in should have permissions set so that files
 1067%           created are only accessible to the current user.
 1068%    - The path must be below 92 *bytes* long (including null terminator) to
 1069%           be portable according to the Linux documentation
 1070%
 1071% tmp_file finds the right /tmp directory, even on Mac OS, so the path is small
 1072% Set 700 (rwx------)  permission so it is only accessible by current user
 1073% Create a secure tmp file in the new directory
 1074% {set,current}_prolog_flag is copied to a thread, so no need to use a mutex.
 1075% Close the stream so sockets can use it
 1076unix_domain_socket_path(Created_Directory, File_Path) :-
 1077    tmp_file(udsock, Created_Directory),
 1078    make_directory(Created_Directory),
 1079    catch(  chmod(Created_Directory, urwx),
 1080            Exception,
 1081            (   catch(delete_directory(Created_Directory), error(_, _), true),
 1082                throw(Exception)
 1083            )
 1084    ),
 1085    setup_call_cleanup( (   current_prolog_flag(tmp_dir, Save_Tmp_Dir),
 1086                            set_prolog_flag(tmp_dir, Created_Directory)
 1087                        ),
 1088                        tmp_file_stream(File_Path, Stream, []),
 1089                        set_prolog_flag(tmp_dir, Save_Tmp_Dir)
 1090                      ),
 1091    close(Stream).
 1092
 1093
 1094% Helper for installing the language_server.pl file to the right
 1095% library directory.
 1096% Call using swipl -s language_server.pl -g "language_server:install_to_library('language_server.pl')" -t halt
 1097install_to_library(File) :-
 1098    once(find_library(Path)),
 1099    copy_file(File, Path),
 1100    make.
 1101
 1102
 1103% Find the base library path, i.e. the one that ends in
 1104% "library/"
 1105find_library(Path) :-
 1106    file_alias_path(library, Path),
 1107    atomic_list_concat(Parts, '/', Path),
 1108    reverse(Parts, Parts_Reverse),
 1109    nth0(0, Parts_Reverse, ''),
 1110    nth0(1, Parts_Reverse, Library),
 1111    string_lower(Library, 'library')