Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch Session to new serialization traits #858

Merged
merged 39 commits into from
Dec 12, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
0b0d02d
RowSerializationContext: Add from_prepared method
Lorak-mmk Nov 11, 2023
18d6393
scylla-cql: Add SerializationError variant for QueryError
Lorak-mmk Nov 8, 2023
ca30092
connection.rs: Don't allow queries with values
Lorak-mmk Nov 8, 2023
dc2b02e
Rename SerializedValues to LegacySerializedValues
Lorak-mmk Dec 5, 2023
d0e8502
scylla-cql: Implement SerializedValues
Lorak-mmk Nov 12, 2023
0ddaa9a
PreparedStatement: Add `serialize_values` helper method
Lorak-mmk Dec 11, 2023
4a18a94
scylla: export scylla_cql::types::serialize
Lorak-mmk Dec 11, 2023
8677601
ClusterData: Switch `get_endpoints` to new serialization API
Lorak-mmk Dec 11, 2023
a523582
ClusterData: switch `compute_token` to new serialization API
Lorak-mmk Dec 11, 2023
de7fc31
PreparedStatement: Switch `calculate_token` to new serialization API
Lorak-mmk Dec 11, 2023
0f63636
PreparedStatement: Switch `compute_partition_key` to new Serializatio…
Lorak-mmk Dec 11, 2023
c724068
RequestSpan: record request sizes lazily
Lorak-mmk Dec 11, 2023
0545568
Session: switch `query_paged` method to new serialization API
Lorak-mmk Dec 11, 2023
50fb19d
Session: switch `query_iter` to new serialization API
Lorak-mmk Dec 11, 2023
615cc33
Session: switch `execute_paged` method to new serialization API
Lorak-mmk Dec 11, 2023
8675646
Session: switch `execute_iter` to new serialization API
Lorak-mmk Dec 11, 2023
5ebb2d2
Session: switch `execute` to new serialization API
Lorak-mmk Dec 11, 2023
9bf7b77
examples: Use both new and old serialization APIs
Lorak-mmk Dec 11, 2023
6cc643a
cql_collections_test: Use both new and old serialization APIs
Lorak-mmk Dec 11, 2023
ea93d39
cql_types_test: Use both new and old serialization APIs
Lorak-mmk Dec 11, 2023
ac88e0a
session_test: Use both new and old serialization APIs
Lorak-mmk Dec 11, 2023
b3261cd
Session: switch `query` method to new serialization API
Lorak-mmk Dec 11, 2023
25a6d36
Book: Derive new serialization traits in udt.md and values.md
Lorak-mmk Dec 12, 2023
c353f17
examples: Remove usage of old serialization APIs
Lorak-mmk Dec 11, 2023
8a33467
cql_collections_test: Remove usage of old serialization APIs
Lorak-mmk Dec 11, 2023
72a81ce
cql_types_test: Remove usage of old serialization APIs
Lorak-mmk Dec 11, 2023
c7e766e
session_test: Remove usage of old serialization macros / traits
Lorak-mmk Dec 11, 2023
4a09baa
CachingSession: Use new serialization APIs
Lorak-mmk Dec 11, 2023
3d32c1c
RowIterator: switch `new_for_prepared_statement` to new serialization…
Lorak-mmk Dec 11, 2023
d3b884b
topology: type-safe serialization in query_filter_keyspace_name
Lorak-mmk Dec 11, 2023
d5b8636
Connection: switch `execute_iter` to new serialization API
Lorak-mmk Dec 11, 2023
a4a27de
RowIterator: switch `new_for_connection_execute_iter` to new serializ…
Lorak-mmk Dec 11, 2023
7ae1abb
Connection: switch `execute` method to new serialization API
Lorak-mmk Dec 11, 2023
5960e3b
Connection: switch `execute_with_consistency` to new serialization API
Lorak-mmk Dec 11, 2023
991ea48
PreparedStatement: switch `extract_partition_key*` and `PrimaryKey::n…
Lorak-mmk Dec 11, 2023
4d5c33a
partitioner: switch `calculate_token_for_partition_key` to new serial…
Lorak-mmk Dec 11, 2023
ee61cb0
QueryParameters: switch `values` to new `SerializedValues`
Lorak-mmk Dec 11, 2023
a35b5b3
scylla: Remove most uses of fallback SerializeRow impls
Lorak-mmk Dec 11, 2023
64c6ac6
Remove unnecessary `#[allow(dead_code)]`
Lorak-mmk Dec 11, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/source/data-types/udt.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,10 @@ Now it can be sent and received just like any other CQL value:
# use std::error::Error;
# async fn check_only_compiles(session: &Session) -> Result<(), Box<dyn Error>> {
use scylla::IntoTypedRows;
use scylla::macros::{FromUserType, IntoUserType};
use scylla::macros::{FromUserType, IntoUserType, SerializeCql};
use scylla::cql_to_rust::FromCqlVal;

#[derive(Debug, IntoUserType, FromUserType)]
#[derive(Debug, IntoUserType, FromUserType, SerializeCql)]
struct MyType {
int_val: i32,
text_val: Option<String>,
Expand Down
4 changes: 2 additions & 2 deletions docs/source/queries/values.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ or a custom struct which derives from `ValueList`.
A few examples:
```rust
# extern crate scylla;
# use scylla::{Session, ValueList, frame::response::result::CqlValue};
# use scylla::{Session, ValueList, SerializeRow, frame::response::result::CqlValue};
# use std::error::Error;
# use std::collections::HashMap;
# async fn check_only_compiles(session: &Session) -> Result<(), Box<dyn Error>> {
Expand All @@ -34,7 +34,7 @@ session

// Sending an integer and a string using a named struct.
// The values will be passed in the order from the struct definition
#[derive(ValueList)]
#[derive(ValueList, SerializeRow)]
struct IntString {
first_col: i32,
second_col: String,
Expand Down
4 changes: 1 addition & 3 deletions examples/compare-tokens.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
use anyhow::Result;
use scylla::frame::value::ValueList;
use scylla::routing::Token;
use scylla::transport::NodeAddr;
use scylla::{Session, SessionBuilder};
Expand Down Expand Up @@ -29,8 +28,7 @@ async fn main() -> Result<()> {
.query("INSERT INTO ks.t (pk) VALUES (?)", (pk,))
.await?;

let serialized_pk = (pk,).serialized()?.into_owned();
let t = prepared.calculate_token(&serialized_pk)?.unwrap().value;
let t = prepared.calculate_token(&(pk,))?.unwrap().value;

println!(
"Token endpoints for query: {:?}",
Expand Down
6 changes: 3 additions & 3 deletions examples/user-defined-type.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use anyhow::Result;
use scylla::macros::{FromUserType, IntoUserType};
use scylla::{IntoTypedRows, Session, SessionBuilder};
use scylla::macros::FromUserType;
use scylla::{IntoTypedRows, SerializeCql, Session, SessionBuilder};
use std::env;

#[tokio::main]
Expand Down Expand Up @@ -29,7 +29,7 @@ async fn main() -> Result<()> {

// Define custom struct that matches User Defined Type created earlier
// wrapping field in Option will gracefully handle null field values
#[derive(Debug, IntoUserType, FromUserType)]
#[derive(Debug, FromUserType, SerializeCql)]
struct MyType {
int_val: i32,
text_val: Option<String>,
Expand Down
6 changes: 3 additions & 3 deletions examples/value_list.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ async fn main() {
.await
.unwrap();

#[derive(scylla::ValueList)]
#[derive(scylla::SerializeRow)]
struct MyType<'a> {
k: i32,
my: Option<&'a str>,
Expand All @@ -36,8 +36,8 @@ async fn main() {
.unwrap();

// You can also use type generics:
#[derive(scylla::ValueList)]
struct MyTypeWithGenerics<S: scylla::frame::value::Value> {
#[derive(scylla::SerializeRow)]
struct MyTypeWithGenerics<S: scylla::serialize::value::SerializeCql> {
k: i32,
my: Option<S>,
}
Expand Down
25 changes: 18 additions & 7 deletions scylla-cql/benches/benchmark.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,17 @@ use std::borrow::Cow;
use criterion::{criterion_group, criterion_main, BenchmarkId, Criterion};

use scylla_cql::frame::request::SerializableRequest;
use scylla_cql::frame::value::SerializedValues;
use scylla_cql::frame::value::ValueList;
use scylla_cql::frame::response::result::ColumnType;
use scylla_cql::frame::{request::query, Compression, SerializedRequest};
use scylla_cql::types::serialize::row::SerializedValues;

fn make_query<'a>(contents: &'a str, values: &'a SerializedValues) -> query::Query<'a> {
fn make_query(contents: &str, values: SerializedValues) -> query::Query<'_> {
query::Query {
contents: Cow::Borrowed(contents),
parameters: query::QueryParameters {
consistency: scylla_cql::Consistency::LocalQuorum,
serial_consistency: None,
values: Cow::Borrowed(values),
values: Cow::Owned(values),
page_size: None,
paging_state: None,
timestamp: None,
Expand All @@ -22,13 +22,24 @@ fn make_query<'a>(contents: &'a str, values: &'a SerializedValues) -> query::Que
}

fn serialized_request_make_bench(c: &mut Criterion) {
let mut values = SerializedValues::new();
let mut group = c.benchmark_group("LZ4Compression.SerializedRequest");
let query_args = [
("INSERT foo INTO ks.table_name (?)", &(1234,).serialized().unwrap()),
("INSERT foo, bar, baz INTO ks.table_name (?, ?, ?)", &(1234, "a value", "i am storing a string").serialized().unwrap()),
("INSERT foo INTO ks.table_name (?)", {
values.add_value(&1234, &ColumnType::Int).unwrap();
values.clone()
}),
("INSERT foo, bar, baz INTO ks.table_name (?, ?, ?)", {
values.add_value(&"a value", &ColumnType::Text).unwrap();
values.add_value(&"i am storing a string", &ColumnType::Text).unwrap();
values.clone()
}),
(
"INSERT foo, bar, baz, boop, blah INTO longer_keyspace.a_big_table_name (?, ?, ?, ?, 1000)",
&(1234, "a value", "i am storing a string", "dc0c8cd7-d954-47c1-8722-a857941c43fb").serialized().unwrap()
{
values.add_value(&"dc0c8cd7-d954-47c1-8722-a857941c43fb", &ColumnType::Text).unwrap();
values.clone()
}
),
];
let queries = query_args.map(|(q, v)| make_query(q, v));
Expand Down
10 changes: 10 additions & 0 deletions scylla-cql/src/errors.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
use crate::frame::frame_errors::{FrameError, ParseError};
use crate::frame::protocol_features::ProtocolFeatures;
use crate::frame::value::SerializeValuesError;
use crate::types::serialize::SerializationError;
use crate::Consistency;
use bytes::Bytes;
use std::io::ErrorKind;
Expand Down Expand Up @@ -340,6 +341,9 @@ pub enum BadQuery {
#[error("Serializing values failed: {0} ")]
SerializeValuesError(#[from] SerializeValuesError),

#[error("Serializing values failed: {0} ")]
SerializationError(#[from] SerializationError),

/// Serialized values are too long to compute partition key
#[error("Serialized values are too long to compute partition key! Length: {0}, Max allowed length: {1}")]
ValuesTooLongForKey(usize, usize),
Expand Down Expand Up @@ -443,6 +447,12 @@ impl From<SerializeValuesError> for QueryError {
}
}

impl From<SerializationError> for QueryError {
fn from(serialized_err: SerializationError) -> QueryError {
QueryError::BadQuery(BadQuery::SerializationError(serialized_err))
}
}
Lorak-mmk marked this conversation as resolved.
Show resolved Hide resolved

impl From<ParseError> for QueryError {
fn from(parse_error: ParseError) -> QueryError {
QueryError::InvalidMessage(format!("Error parsing message: {}", parse_error))
Expand Down
3 changes: 3 additions & 0 deletions scylla-cql/src/frame/frame_errors.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
use super::response;
use crate::cql_to_rust::CqlTypeError;
use crate::frame::value::SerializeValuesError;
use crate::types::serialize::SerializationError;
use thiserror::Error;

#[derive(Error, Debug)]
Expand Down Expand Up @@ -44,5 +45,7 @@ pub enum ParseError {
#[error(transparent)]
SerializeValuesError(#[from] SerializeValuesError),
#[error(transparent)]
SerializationError(#[from] SerializationError),
#[error(transparent)]
CqlTypeError(#[from] CqlTypeError),
}
8 changes: 4 additions & 4 deletions scylla-cql/src/frame/request/batch.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ use crate::frame::{
frame_errors::ParseError,
request::{RequestOpcode, SerializableRequest},
types::{self, SerialConsistency},
value::{BatchValues, BatchValuesIterator, SerializedValues},
value::{BatchValues, BatchValuesIterator, LegacySerializedValues},
};

use super::DeserializableRequest;
Expand Down Expand Up @@ -186,7 +186,7 @@ impl<'s, 'b> From<&'s BatchStatement<'b>> for BatchStatement<'s> {
}
}

impl<'b> DeserializableRequest for Batch<'b, BatchStatement<'b>, Vec<SerializedValues>> {
impl<'b> DeserializableRequest for Batch<'b, BatchStatement<'b>, Vec<LegacySerializedValues>> {
fn deserialize(buf: &mut &[u8]) -> Result<Self, ParseError> {
let batch_type = buf.get_u8().try_into()?;

Expand All @@ -196,7 +196,7 @@ impl<'b> DeserializableRequest for Batch<'b, BatchStatement<'b>, Vec<SerializedV
let batch_statement = BatchStatement::deserialize(buf)?;

// As stated in CQL protocol v4 specification, values names in Batch are broken and should be never used.
let values = SerializedValues::new_from_frame(buf, false)?;
let values = LegacySerializedValues::new_from_frame(buf, false)?;

Ok((batch_statement, values))
})
Expand Down Expand Up @@ -233,7 +233,7 @@ impl<'b> DeserializableRequest for Batch<'b, BatchStatement<'b>, Vec<SerializedV
.then(|| types::read_long(buf))
.transpose()?;

let (statements, values): (Vec<BatchStatement>, Vec<SerializedValues>) =
let (statements, values): (Vec<BatchStatement>, Vec<LegacySerializedValues>) =
statements_with_values.into_iter().unzip();

Ok(Self {
Expand Down
21 changes: 11 additions & 10 deletions scylla-cql/src/frame/request/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ pub use startup::Startup;
use self::batch::BatchStatement;

use super::types::SerialConsistency;
use super::value::SerializedValues;
use super::value::LegacySerializedValues;

#[derive(Debug, Copy, Clone, PartialEq, Eq, PartialOrd, Ord, TryFromPrimitive)]
#[repr(u8)]
Expand Down Expand Up @@ -59,7 +59,7 @@ pub trait DeserializableRequest: SerializableRequest + Sized {
pub enum Request<'r> {
Query(Query<'r>),
Execute(Execute<'r>),
Batch(Batch<'r, BatchStatement<'r>, Vec<SerializedValues>>),
Batch(Batch<'r, BatchStatement<'r>, Vec<LegacySerializedValues>>),
}

impl<'r> Request<'r> {
Expand Down Expand Up @@ -112,9 +112,10 @@ mod tests {
query::{Query, QueryParameters},
DeserializableRequest, SerializableRequest,
},
response::result::ColumnType,
types::{self, SerialConsistency},
value::SerializedValues,
},
types::serialize::row::SerializedValues,
Consistency,
};

Expand All @@ -130,7 +131,7 @@ mod tests {
paging_state: Some(vec![2, 1, 3, 7].into()),
values: {
let mut vals = SerializedValues::new();
vals.add_value(&2137).unwrap();
vals.add_value(&2137, &ColumnType::Int).unwrap();
Cow::Owned(vals)
},
};
Expand All @@ -157,8 +158,8 @@ mod tests {
paging_state: None,
values: {
let mut vals = SerializedValues::new();
vals.add_named_value("the_answer", &42).unwrap();
vals.add_named_value("really?", &2137).unwrap();
vals.add_value(&42, &ColumnType::Int).unwrap();
vals.add_value(&2137, &ColumnType::Int).unwrap();
Cow::Owned(vals)
},
};
Expand Down Expand Up @@ -189,8 +190,8 @@ mod tests {

// Not execute's values, because named values are not supported in batches.
values: vec![
query.parameters.values.deref().clone(),
query.parameters.values.deref().clone(),
query.parameters.values.deref().to_old_serialized_values(),
query.parameters.values.deref().to_old_serialized_values(),
],
};
{
Expand All @@ -212,7 +213,7 @@ mod tests {
timestamp: None,
page_size: None,
paging_state: None,
values: Cow::Owned(SerializedValues::new()),
values: Cow::Borrowed(SerializedValues::EMPTY),
};
let query = Query {
contents: contents.clone(),
Expand Down Expand Up @@ -261,7 +262,7 @@ mod tests {
serial_consistency: None,
timestamp: None,

values: vec![query.parameters.values.deref().clone()],
values: vec![query.parameters.values.deref().to_old_serialized_values()],
};
{
let mut buf = Vec::new();
Expand Down
18 changes: 11 additions & 7 deletions scylla-cql/src/frame/request/query.rs
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
use std::borrow::Cow;

use crate::frame::{frame_errors::ParseError, types::SerialConsistency};
use crate::{
frame::{frame_errors::ParseError, types::SerialConsistency},
types::serialize::row::SerializedValues,
};
use bytes::{Buf, BufMut, Bytes};

use crate::{
frame::request::{RequestOpcode, SerializableRequest},
frame::types,
frame::value::SerializedValues,
};

use super::DeserializableRequest;
Expand Down Expand Up @@ -102,10 +104,6 @@ impl QueryParameters<'_> {
flags |= FLAG_WITH_DEFAULT_TIMESTAMP;
}

if self.values.has_names() {
flags |= FLAG_WITH_NAMES_FOR_VALUES;
}

buf.put_u8(flags);

if !self.values.is_empty() {
Expand Down Expand Up @@ -151,8 +149,14 @@ impl<'q> QueryParameters<'q> {
let default_timestamp_flag = (flags & FLAG_WITH_DEFAULT_TIMESTAMP) != 0;
let values_have_names_flag = (flags & FLAG_WITH_NAMES_FOR_VALUES) != 0;

if values_have_names_flag {
return Err(ParseError::BadIncomingData(
"Named values in frame are currently unsupported".to_string(),
));
}

let values = Cow::Owned(if values_flag {
SerializedValues::new_from_frame(buf, values_have_names_flag)?
SerializedValues::new_from_frame(buf)?
} else {
SerializedValues::new()
});
Expand Down
Loading