Skip to content

Commit 40b9dba

Browse files
pmishchenko-uaduwenxin99
pmishchenko-ua
and
duwenxin99
authored
feat(singlestore): Add SingleStore Source and Tools (googleapis#1333)
## Description --- - This PR adds SingleStore database source and tools. The code is mostly based on MySQL source and tools, and it uses the same go-mysql driver. - https://github.com/singlestore-labs/singlestoredb-dev-image can be used to deploy a test SingleStore instance. In this PR the default port is set to 3308 so the command would be ```docker run \ -d --name singlestoredb-dev \ -e ROOT_PASSWORD="YOUR SINGLESTORE ROOT PASSWORD" \ -p 3308:3306 ghcr.io/singlestore-labs/singlestoredb-dev:latest ``` ## PR Checklist --- > Thank you for opening a Pull Request! Before submitting your PR, there are a > few things you can do to make sure it goes smoothly: - [x] Make sure you reviewed [CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md) - [x] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/langchain-google-alloydb-pg-python/issues/new/choose) before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea - [x] Ensure the tests and linter pass - [x] Code coverage does not decrease (if any source code was changed) - [x] Appropriate docs were updated (if necessary) - [x] Make sure to add `!` if this involve a breaking change 🛠️ Fixes googleapis#1348 --------- Co-authored-by: Wenxin Du <117315983+duwenxin99@users.noreply.github.com>
1 parent 47b66d3 commit 40b9dba

15 files changed

Lines changed: 1664 additions & 0 deletions

File tree

‎.ci/integration.cloudbuild.yaml‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -784,6 +784,28 @@ steps:
784784
"Serverless Spark" \
785785
serverlessspark
786786
787+
- id: "singlestore"
788+
name: golang:1
789+
waitFor: ["compile-test-binary"]
790+
entrypoint: /bin/bash
791+
env:
792+
- "GOPATH=/gopath"
793+
- "SINGLESTORE_PORT=$_SINGLESTORE_PORT"
794+
- "SINGLESTORE_USER=$_SINGLESTORE_USER"
795+
- "SINGLESTORE_DATABASE=$_SINGLESTORE_DATABASE"
796+
- "SERVICE_ACCOUNT_EMAIL=$SERVICE_ACCOUNT_EMAIL"
797+
secretEnv: ["SINGLESTORE_PASSWORD", "SINGLESTORE_HOST", "CLIENT_ID"]
798+
volumes:
799+
- name: "go"
800+
path: "/gopath"
801+
args:
802+
- -c
803+
- |
804+
.ci/test_with_coverage.sh \
805+
"SingleStore" \
806+
singlestore \
807+
singlestore
808+
787809
availableSecrets:
788810
secretManager:
789811
- versionName: projects/$PROJECT_ID/secrets/cloud_sql_pg_user/versions/latest
@@ -892,6 +914,10 @@ availableSecrets:
892914
env: ORACLE_PASS
893915
- versionName: projects/$PROJECT_ID/secrets/oracle_host/versions/latest
894916
env: ORACLE_HOST
917+
- versionName: projects/$PROJECT_ID/secrets/singlestore_pass/versions/latest
918+
env: SINGLESTORE_PASSWORD
919+
- versionName: projects/$PROJECT_ID/secrets/singlestore_host/versions/latest
920+
env: SINGLESTORE_HOST
895921

896922
options:
897923
logging: CLOUD_LOGGING_ONLY
@@ -947,3 +973,7 @@ substitutions:
947973
_YUGABYTEDB_PORT: "5433"
948974
_YUGABYTEDB_LOADBALANCE: "false"
949975
_ORACLE_SERVER_NAME: "FREEPDB1"
976+
_SINGLESTORE_HOST: 127.0.0.1
977+
_SINGLESTORE_PORT: "3308"
978+
_SINGLESTORE_DATABASE: "singlestore"
979+
_SINGLESTORE_USER: "root"

‎cmd/root.go‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -168,6 +168,8 @@ import (
168168
_ "github.com/googleapis/genai-toolbox/internal/tools/serverlessspark/serverlesssparkcancelbatch"
169169
_ "github.com/googleapis/genai-toolbox/internal/tools/serverlessspark/serverlesssparkgetbatch"
170170
_ "github.com/googleapis/genai-toolbox/internal/tools/serverlessspark/serverlesssparklistbatches"
171+
_ "github.com/googleapis/genai-toolbox/internal/tools/singlestore/singlestoreexecutesql"
172+
_ "github.com/googleapis/genai-toolbox/internal/tools/singlestore/singlestoresql"
171173
_ "github.com/googleapis/genai-toolbox/internal/tools/spanner/spannerexecutesql"
172174
_ "github.com/googleapis/genai-toolbox/internal/tools/spanner/spannerlisttables"
173175
_ "github.com/googleapis/genai-toolbox/internal/tools/spanner/spannersql"
@@ -212,6 +214,7 @@ import (
212214
_ "github.com/googleapis/genai-toolbox/internal/sources/postgres"
213215
_ "github.com/googleapis/genai-toolbox/internal/sources/redis"
214216
_ "github.com/googleapis/genai-toolbox/internal/sources/serverlessspark"
217+
_ "github.com/googleapis/genai-toolbox/internal/sources/singlestore"
215218
_ "github.com/googleapis/genai-toolbox/internal/sources/spanner"
216219
_ "github.com/googleapis/genai-toolbox/internal/sources/sqlite"
217220
_ "github.com/googleapis/genai-toolbox/internal/sources/tidb"

‎docs/en/resources/sources/singlestore.md‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
---
2+
title: "SingleStore"
3+
type: docs
4+
weight: 1
5+
description: >
6+
SingleStore is the cloud-native database built with speed and scale to power data-intensive applications.
7+
---
8+
9+
## About
10+
11+
[SingleStore][singlestore-docs] is a distributed SQL database built to power intelligent applications. It is both relational and multi-model, enabling developers to easily build and scale applications and workloads.
12+
13+
SingleStore is built around Universal Storage which combines in-memory rowstore and on-disk columnstore data formats to deliver a single table type that is optimized to handle both transactional and analytical workloads.
14+
15+
[singlestore-docs]: https://docs.singlestore.com/
16+
17+
## Available Tools
18+
19+
- [`singlestore-sql`](../tools/singlestore/singlestore-sql.md)
20+
Execute pre-defined prepared SQL queries in SingleStore.
21+
22+
- [`singlestore-execute-sql`](../tools/singlestore/singlestore-execute-sql.md)
23+
Run parameterized SQL queries in SingleStore.
24+
25+
## Requirements
26+
27+
### Database User
28+
29+
This source only uses standard authentication. You will need to [create a
30+
database user][singlestore-user] to login to the database with.
31+
32+
[singlestore-user]: https://docs.singlestore.com/cloud/reference/sql-reference/security-management-commands/create-user/
33+
34+
## Example
35+
36+
```yaml
37+
sources:
38+
my-singlestore-source:
39+
kind: singlestore
40+
host: 127.0.0.1
41+
port: 3306
42+
database: my_db
43+
user: ${USER_NAME}
44+
password: ${PASSWORD}
45+
queryTimeout: 30s # Optional: query timeout duration
46+
```
47+
48+
{{< notice tip >}}
49+
Use environment variable replacement with the format ${ENV_NAME}
50+
instead of hardcoding your secrets into the configuration file.
51+
{{< /notice >}}
52+
53+
## Reference
54+
55+
| **field** | **type** | **required** | **description** |
56+
| ------------ | :------: | :----------: | ----------------------------------------------------------------------------------------------- |
57+
| kind | string | true | Must be "singlestore". |
58+
| host | string | true | IP address to connect to (e.g. "127.0.0.1"). |
59+
| port | string | true | Port to connect to (e.g. "3306"). |
60+
| database | string | true | Name of the SingleStore database to connect to (e.g. "my_db"). |
61+
| user | string | true | Name of the SingleStore database user to connect as (e.g. "admin"). |
62+
| password | string | true | Password of the SingleStore database user. |
63+
| queryTimeout | string | false | Maximum time to wait for query execution (e.g. "30s", "2m"). By default, no timeout is applied. |

‎docs/en/resources/tools/singlestore/_index.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
title: "SingleStore"
3+
type: docs
4+
weight: 1
5+
description: >
6+
Tools that work with SingleStore Sources
7+
---

‎docs/en/resources/tools/singlestore/singlestore-execute-sql.md‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
---
2+
title: "singlestore-execute-sql"
3+
type: docs
4+
weight: 1
5+
description: >
6+
A "singlestore-execute-sql" tool executes a SQL statement against a SingleStore
7+
database.
8+
aliases:
9+
- /resources/tools/singlestore-execute-sql
10+
---
11+
12+
## About
13+
14+
A `singlestore-execute-sql` tool executes a SQL statement against a SingleStore
15+
database. It's compatible with the following sources:
16+
17+
- [singlestore](../../sources/singlestore.md)
18+
19+
`singlestore-execute-sql` takes one input parameter `sql` and runs the sql
20+
statement against the `source`.
21+
22+
> **Note:** This tool is intended for developer assistant workflows with
23+
> human-in-the-loop and shouldn't be used for production agents.
24+
25+
## Example
26+
27+
```yaml
28+
tools:
29+
execute_sql_tool:
30+
kind: singlestore-execute-sql
31+
source: my-s2-instance
32+
description: Use this tool to execute sql statement
33+
```
34+
35+
## Reference
36+
37+
| **field** | **type** | **required** | **description** |
38+
|-------------|:------------------------------------------:|:------------:|--------------------------------------------------------------------------------------------------|
39+
| kind | string | true | Must be "singlestore-execute-sql". |
40+
| source | string | true | Name of the source the SQL should execute on. |
41+
| description | string | true | Description of the tool that is passed to the LLM. |

‎docs/en/resources/tools/singlestore/singlestore-sql.md‎

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
---
2+
title: "singlestore-sql"
3+
type: docs
4+
weight: 1
5+
description: >
6+
A "singlestore-sql" tool executes a pre-defined SQL statement against a SingleStore
7+
database.
8+
aliases:
9+
- /resources/tools/singlestore-sql
10+
---
11+
12+
## About
13+
14+
A `singlestore-execute-sql` tool executes a SQL statement against a SingleStore
15+
database. It's compatible with the following sources:
16+
17+
- [singlestore](../../sources/singlestore.md)
18+
19+
The specified SQL statement expects parameters in the SQL query to be in the form of placeholders `?`.
20+
21+
## Example
22+
23+
> **Note:** This tool uses parameterized queries to prevent SQL injections.
24+
> Query parameters can be used as substitutes for arbitrary expressions.
25+
> Parameters cannot be used as substitutes for identifiers, column names, table
26+
> names, or other parts of the query.
27+
28+
```yaml
29+
tools:
30+
search_flights_by_number:
31+
kind: singlestore-sql
32+
source: my-s2-instance
33+
statement: |
34+
SELECT * FROM flights
35+
WHERE airline = ?
36+
AND flight_number = ?
37+
LIMIT 10
38+
description: |
39+
Use this tool to get information for a specific flight.
40+
Takes an airline code and flight number and returns info on the flight.
41+
Do NOT use this tool with a flight id. Do NOT guess an airline code or flight number.
42+
A airline code is a code for an airline service consisting of two-character
43+
airline designator and followed by flight number, which is 1 to 4 digit number.
44+
For example, if given CY 0123, the airline is "CY", and flight_number is "123".
45+
Another example for this is DL 1234, the airline is "DL", and flight_number is "1234".
46+
If the tool returns more than one option choose the date closes to today.
47+
Example:
48+
{{
49+
"airline": "CY",
50+
"flight_number": "888",
51+
}}
52+
Example:
53+
{{
54+
"airline": "DL",
55+
"flight_number": "1234",
56+
}}
57+
parameters:
58+
- name: airline
59+
type: string
60+
description: Airline unique 2 letter identifier
61+
- name: flight_number
62+
type: string
63+
description: 1 to 4 digit number
64+
```
65+
66+
### Example with Template Parameters
67+
68+
> **Note:** This tool allows direct modifications to the SQL statement,
69+
> including identifiers, column names, and table names. **This makes it more
70+
> vulnerable to SQL injections**. Using basic parameters only (see above) is
71+
> recommended for performance and safety reasons. For more details, please check
72+
> [templateParameters](..#template-parameters).
73+
74+
```yaml
75+
tools:
76+
list_table:
77+
kind: singlestore-sql
78+
source: my-s2-instance
79+
statement: |
80+
SELECT * FROM {{.tableName}};
81+
description: |
82+
Use this tool to list all information from a specific table.
83+
Example:
84+
{{
85+
"tableName": "flights",
86+
}}
87+
templateParameters:
88+
- name: tableName
89+
type: string
90+
description: Table to select from
91+
```
92+
93+
## Reference
94+
95+
| **field** | **type** | **required** | **description** |
96+
|--------------------|:------------------------------------------------:|:------------:|--------------------------------------------------------------------------------------------------------------------------------------------|
97+
| kind | string | true | Must be "singlestore-sql". |
98+
| source | string | true | Name of the source the SQL should execute on. |
99+
| description | string | true | Description of the tool that is passed to the LLM. |
100+
| statement | string | true | SQL statement to execute on. |
101+
| parameters | [parameters](../#specifying-parameters) | false | List of [parameters](../#specifying-parameters) that will be inserted into the SQL statement. |
102+
| templateParameters | [templateParameters](..#template-parameters) | false | List of [templateParameters](..#template-parameters) that will be inserted into the SQL statement before executing prepared statement. |

‎internal/prebuiltconfigs/prebuiltconfigs_test.go‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,7 @@ var expectedToolSources = []string{
4747
"oceanbase",
4848
"postgres",
4949
"serverless-spark",
50+
"singlestore",
5051
"spanner-postgres",
5152
"spanner",
5253
"sqlite",
@@ -118,6 +119,7 @@ func TestGetPrebuiltTool(t *testing.T) {
118119
mssql_config, _ := Get("mssql")
119120
oceanbase_config, _ := Get("oceanbase")
120121
postgresconfig, _ := Get("postgres")
122+
singlestore_config, _ := Get("singlestore")
121123
spanner_config, _ := Get("spanner")
122124
spannerpg_config, _ := Get("spanner-postgres")
123125
mindsdb_config, _ := Get("mindsdb")
@@ -190,6 +192,9 @@ func TestGetPrebuiltTool(t *testing.T) {
190192
if len(postgresconfig) <= 0 {
191193
t.Fatalf("unexpected error: could not fetch postgres prebuilt tools yaml")
192194
}
195+
if len(singlestore_config) <= 0 {
196+
t.Fatalf("unexpected error: could not fetch singlestore prebuilt tools yaml")
197+
}
193198
if len(spanner_config) <= 0 {
194199
t.Fatalf("unexpected error: could not fetch spanner prebuilt tools yaml")
195200
}

0 commit comments

Comments
 (0)