sql: support prepared statements

[vr009]

What has been done? Why? What problem is being solved?

This patch adds the support of prepared statements.
Added a new type for handling prepared statements.
Added new methods for connector's interface in connector.go.
Added new IPROTO-constants for support of prepared statements
in const.go. Updated multi-package for corresponding it to connector's
interface.

Added a new function for checking the count of prepared statements in
config.lua in tests for Prepare and Unprepare.
Added benchmarks for SQL-select prepared statement.
Added examples of using Prepare in example_test.go.
Fixed some grammar inconsistencies for the method Execute.
Updated CHANGELOG.md.

I didn't forget about (remove if it is not applicable):

• Tests (see documentation for a testing package)
• Changelog (see documentation for changelog format)
• Documentation (see documentation for documentation style guide)

Related issues:

Follows up #62
Closes #117

Considered variants of API

v1

Prepared Statement is just an object only having a constructor.
All requests get constructed with request objects.

Single connection usage:

    conn, err := tarantool.Connect(server, opts)
	
    prepareReq := tarantool.NewPrepareRequest("SQL statement")
    prepareResp, err := conn.Do(prepareReq).Get()

    stmt := tarantool.NewPreparedStatement(conn, prepareResp)
	
    execPreparedReq := tarantool.NewPreparedExecuteRequest(stmt).Args([]interface{}{1, "test"})
    unprepareReq := tarantool.NewUnprepareRequest(stmt)
	
    resp, err = conn.Do(execPreparedReq).Get()
    resp, err = conn.Do(unprepareReq).Get()

For connection pool it is complicated to use without any method that returns
a connection from pool explicitly.

Consider the example:

    pool, err := examplePool(Roles)
	
    preapreReq := tarantool.NewPrepareRequest("SELECT 1")
    prepareResp, err := pool.Do(preapreReq, connection_pool.ANY).Get()
	
	// We cannot know what for a connection is used to prepare a statement.
	// There is need to save it anywhere, the response object is not suited
	// cause it is an object that returns only after synchronous call .Get()
	// so it breaks the scenario of using Future objects
	
    stmt := connection_pool.NewPreparedStatement(pool, prepareResp)
	
    execPreparedReq := tarantool.NewPreparedExecuteRequest(stmt).Args([]interface{}{1, "test"})
    unprepareReq := tarantool.NewUnprepareRequest(stmt)

    resp, err := pool.Do(execPreparedReq, connection_pool.ANY).Get()
    resp, err := pool.Do(unprepareReq, connection_pool.ANY).Get()

Implementation:

type PreparedStatement struct {
    StatementID PreparedStatementID
    MetaData    []ColumnMetaData
    ParamCount  uint64
    Conn        *Connection
}

func NewPreparedStatement(conn *Connection, resp *Response) *PreparedStatement {
    stmt := new(PreparedStatement)
    stmt.Conn = conn
    stmt.ParamCount = resp.BindCount
    stmt.MetaData = resp.MetaData
    stmt.StatementID = PreparedStatementID(resp.StmtID)
    return stmt
}

Advantages:

• unified approach to all requests
• extendable (context support is easy to provide)

Disadvantages:

• tonnes of code
• a lot of preparation
• need to workaround connection_pool problem

v2

Prepared Statement object is mapped from lua interface
with specific extensions for connectors interface.
It has own methods for execute and unprepare actions.

Connector interface:

type Connector interface {
    ...
    Prepare(expr string) (stmt *PreparedStatement, err error)
    PrepareAsync(expr string) *PreparedStatement
    Do(req Request) (fut *Future)
    ...
}

Single connection usage:

    client, err := tarantool.Connect(server, opts)
	// pass a query to prepare
    stmtResp, err := client.Prepare("SELECT id FROM SQL_TEST WHERE id=? AND name=?")
	// pass the id of the statement to Execute
    resp, err := stmtResp.Execute([]interface{}{1, "test_1"})
	// use the same object for execute with other arguments
    resp, err = stmtResp.Execute([]interface{}{2, "test_2"})

Connection pool usage:

    pool, err := examplePool(Roles)
    // pass a query to prepare
    stmtResp, err := pool.Prepare("SELECT id FROM SQL_TEST WHERE id=? AND name=?")
    // pass the id of the statement to Execute
    resp, err := stmtResp.Execute([]interface{}{1, "test_1"})
    // use the same object for execute with other arguments
    resp, err = stmtResp.Execute([]interface{}{2, "test_2"})

Implementation:

request.go:

type PreparedStatement struct {
    StatementID PreparedStatementID
    MetaData    []ColumnMetaData
    ParamCount  uint64
    conn        *Connection
    fut         *Future
}

func newPreparedStatement(fut *Future, conn *Connection) *PreparedStatement {
    stmt := new(PreparedStatement)
    stmt.fut = fut
    stmt.conn = conn
    return stmt
}

// wait until the prepared statement is ready and fill the statement object
func (stmt *PreparedStatement) wait() error {
    resp, err := stmt.fut.Get()
    stmt.StatementID = PreparedStatementID(resp.StmtID)
    stmt.MetaData = resp.MetaData
    stmt.ParamCount = resp.BindCount
    return err
}

// UnprepareAsync sends an undo request and returns Future
func (stmt *PreparedStatement) UnprepareAsync() *Future {
    err := stmt.wait()
    if err != nil {
        errFut := &Future{err: err}
        return errFut
    }
    req := newUnprepareRequest(*stmt)
    fut := stmt.conn.Do(req)
    return fut
}

// Unprepare undo the prepared statement
func (stmt *PreparedStatement) Unprepare() (resp *Response, err error) {
    return stmt.UnprepareAsync().Get()
}

// ExecuteAsync sends the prepared SQL statement for execution and returns Future
func (stmt *PreparedStatement) ExecuteAsync(args interface{}) *Future {
    err := stmt.wait()
    if err != nil {
        errFut := &Future{err: err}
        return errFut
    }
    req := newPreparedExecuteRequest(*stmt)
    req.Args(args)
    fut := stmt.conn.Do(req)
    return fut
}

// Execute sends the prepared SQL statement for execution
func (stmt *PreparedStatement) Execute(args interface{}) (resp *Response, err error) {
    return stmt.ExecuteAsync(args).Get()
}

connection.go:

func (conn *Connection) PrepareAsync(expr string) *PreparedStatement {
    req := newPrepareRequest(expr)
    fut := conn.Do(req)
    stmt := newPreparedStatement(fut, conn)
    return stmt
}

func (conn *Connection) Prepare(expr string) (stmt *PreparedStatement, err error) {
    stmt = conn.PrepareAsync(expr)
    err = stmt.wait()
    return stmt, err
}

Advantages;

• less code for use
• expressive
• pretty close to Lua api
• easy to solve connection_pool problem

Disadvantages:

• inconsistent API
• looks more difficult to extend it (for context support we need to bring more inconsistency to API - very BAD)
• not using Future objects

v3 - the final one

Try to mix both approaches.
We leave the option for constructing prepared statement manually with request objects.
(But it doesn't work well with connection_pool without some method like pool.GetConnectionFromPool())
We give a user more friendly API with wrappers for request objects.

Connector interface:

type Connector interface {
    ...
    Do(req Request) (fut *Future)
    ...
}

Single connection Usage

Single connection usage:

    client, err := tarantool.Connect(server, opts)
    // pass a query to prepare
    stmt, err := client.NewPreparedStatement("SELECT id FROM SQL_TEST WHERE id=? AND name=?")
	
    execReq := tarantool.NewExecuteRequest(stmt)
            .Args([]interface{}{})
    	    .Context(context.Background())
	
    unprepareReq := tarantool.NewUnprepareRequest(stmt)
        .Context(context.Background())
	
    resp, err := client.Do(execReq).Get()
    resp, err = client.Do(unpreapreReq).Get()

Connection pool usage:

    pool, err := examplePool(Roles)
	
    stmt, err := pool.NewPreparedStatement("SELECT id FROM SQL_TEST WHERE id=? AND name=?")
    
    execReq := pool.NewExecuteRequest(stmt)
        .Args([]interface{}{})
	.Context(context.Background())
    
    unprepareReq := pool.NewUnprepareRequest(stmt)
        .Context(context.Background())
    
    resp, err := pool.Do(execReq).Get()
    resp, err = pool.Do(unpreapreReq).Get()

Advantages:

• easy to use
• ability to use futures if it is needed

Disadvantages:

• inconsistant with new Do-API
• only synchronous prepare call

v4 - the proposed one

Purposes:

1. Make a same interface. All requests have a Connection.Request, Connection.RequestAsync and Connection.RequestTyped calls.
2. Request object support. To have support of context at least, extensibility.
3. It should work for Connection and ConnectionPool.

A public API proposal:

// PreparedStatement

type PreparedStatement {
    Conn               Connection
    StatementID        PreparedStatementID
    MetaData           []ColumnMetaData
    ParamCount         uint64
}

func NewPreparedStatement(conn Connection, StatementID PreparedStatementID, MetaData []ColumntMetaData, ParamCount uint64) *PreparedStatement {}
func NewPreparedStatementFromResponse(conn Connection, resp Response) *PreparedStatement {}

// Connection

func (conn *Connection) Prepare(expr string) (resp *Response, err error)
func (conn *Connection) PrepareTyped(expr string, stmt interface{}) err error
func (conn *Connection) PrepareAsync(expr string) *Future

// Request objects

type PrepareRequest {
	baseRequest
}

type PreparedStatementRequest interface {
	Request
	GetPreparedStatement() *PreparedStatement 
}

type ExecutePreparedStatementRequest {
	PrepearedStatementRequest
}

type UnpreparePreparedStatementRequest {
	PrepearedStatementRequest
}

Usage example:

stmt = conn.Do(NewPrepareStatementRequest("statement string")).Get().Data.(PreparedStatement)
conn.Do(NewExecutePrepareStatementRequest(stmt))
conn.Do(NewUnpreparePreparedStatementRequest(stmt))

stmt = pool.Do(NewPrepareStatementRequest("statement string")).Get().Data.(PreparedStatement)
pool.Do(NewExecutePrepareStatementRequest(stmt, args))
pool.Do(NewUnpreparePreparedStatementRequest(stmt))

• We can add a wrappers for PrepareStatement. it's a bit redundant, but it's looks like a Lua interface:

func (stmt *PreparedStatement) Execute(args interface{}) (resp Response, err error)
func (stmt *PreparedStatement) ExecuteTyped(args interface{}, tup interface{}) (err error)
func (stmt *PreparedStatement) ExecuteAsync(args interface{}) (*Future)

func (stmt *PreparedStatement) Unprepare() (resp Response, err error)
func (stmt *PreparedStatement) UnprepareTyped(tup interface{}) (err error)
func (stmt *PreparedStatement) UnprepareAsync() (*Future)

In addition, we can replace PreparedStatement with a shortest naming. Maybe just Prepared?

See:

1. https://www.tarantool.io/ru/doc/latest/reference/reference_lua/box_sql/prepare/

[vr009]

I am not sure about this part of API:

func (conn *Connection) Prepare(expr string) (resp *Response, err error)

Maybe we should return uint64 as the first value? Just for getting the statement id. Like this one:

func (conn *Connection) Prepare(expr string) (uint64, err error)

The 1st option is considered due to its consistency with other connector's methods.

[oleg-jukovec]

I am not sure about this part of API:

func (conn *Connection) Prepare(expr string) (resp *Response, err error)

Maybe we should return uint64 as the first value? Just for getting the statement id. Like this one:

func (conn *Connection) Prepare(expr string) (uint64, err error)

The 1st option is considered due to its consistency with other connector's methods.

What can be useful to do with a response? If the response is needed only to get the StmtID, then it is better to choose the second one.

Also, would it be better to create PreparedStatement (or other naming) type as alias for uint64?

[oleg-jukovec]

Thank you for the pull request. It seems to me that there is a missing method Unprepare. Also, I don't like using the method Connection.Execute for two cases based on an argument type (but it less critical).

[oleg-jukovec]

Good news! You can rebase to the master branch.

[oleg-jukovec]

Thank you for your work! It seems to me that now the API is not consistent with request objects API. Could you do that, please?

[oleg-jukovec]

Purposes:

1. Make a same interface. All requests have a Connection.Request, Connection.RequestAsync and Connection.RequestTyped calls.
2. Request object support. To have support of context at least, extensibility.
3. It should work for Connection and ConnectionPool.

A public API proposal:

// PreparedStatement

type PreparedStatement {
    Conn               Connection
    StatementID        PreparedStatementID
    MetaData           []ColumnMetaData
    ParamCount         uint64
}

func NewPreparedStatement(conn Connection, StatementID PreparedStatementID, MetaData []ColumntMetaData, ParamCount uint64) (*PreparedStatement, error)
func NewPreparedStatementFromResponse(resp Response) *PreparedStatement

// Connection

func (conn *Connection) Prepare(expr string) (resp *Response, err error)
func (conn *Connection) PrepareTyped(expr string, stmt interface{}) err error // ?
func (conn *Connection) PrepareAsync(expr string) *Future

// Request objects

type PrepareRequest {
	baseRequest
}

type PreparedStatementRequest interface {
	Request
	GetPreparedStatement() *PreparedStatement 
}

type ExecutePreparedStatementRequest {
	PrepearedStatementRequest
}

type UnpreparePreparedStatementRequest {
	PrepearedStatementRequest
}

Usage example:

// stmt := conn.Do(NewPrepareRequest("statement string")).Get().Data.(PreparedStatement)
resp, err := conn.Do(NewPrepareRequest("statement string")).Get()
stmt, err := NewPreparedStatementFromResponse(resp)
conn.Do(NewExecutePrepareStatementRequest(stmt))
conn.Do(NewUnpreparePreparedStatementRequest(stmt))

// stmt := conn.Do(NewPrepareRequest("statement string")).Get().Data.(PreparedStatement)
resp, err := pool.Do(NewPrepareRequest("statement string")).Get()
stmt, err := NewPreparedStatementFromResponse(resp)
pool.Do(NewExecutePrepareStatementRequest(stmt, args))
pool.Do(NewUnpreparePreparedStatementRequest(stmt))

We can create a PreparedStatement object here:

go-tarantool/response.go

Lines 68 to 69 in 988edce

	}
	return nil

We can add a wrappers for PreparedStatement. it's a bit redundant, but it's looks like a Lua interface:

func (stmt *PreparedStatement) Execute(args interface{}) (resp Response, err error)
func (stmt *PreparedStatement) ExecuteTyped(args interface{}, tup interface{}) (err error)
func (stmt *PreparedStatement) ExecuteAsync(args interface{}) (*Future)

func (stmt *PreparedStatement) Unprepare() (resp Response, err error)
func (stmt *PreparedStatement) UnprepareTyped(tup interface{}) (err error) // ?
func (stmt *PreparedStatement) UnprepareAsync() (*Future)

In addition, we can replace PreparedStatement with a shortest naming. Maybe just Prepared?

As an alternative we can add a method:

func (stmt *PreparedStatement) Do(req PreparedStatementRequest) (*Future)

The approach (only Connection.Do, or ConnectionDo() + PreparedStatement.Do()) should be synchronized with #185 .

See:

1. https://www.tarantool.io/ru/doc/latest/reference/reference_lua/box_sql/prepare/

[oleg-jukovec]

I have no critical complaints. Thanks for the great work done!

LGTM after resolving all my active conversations.

[oleg-jukovec]

We're almost there, final push!

[oleg-jukovec]

Thank you for the great work done!

I apologize for being too picky, and not being accurate in my suggestions. I will try to improve the review process in the future. But I believe we have achieved the goal - users will like the result.

[vr009]

I apologize for being too picky, and not being accurate in my suggestions. I will try to improve the review process in the future. But I believe we have achieved the goal - users will like the result.

Thank you for detailed review!
I apologize for my inattention to some obvious places, I promise to improve self-review too.

[AnaNek]

LGTM!