Skip to content

Commit

Permalink
wip
Browse files Browse the repository at this point in the history
  • Loading branch information
LironMShemen committed Dec 22, 2024
1 parent 2f60cd4 commit a03e89d
Show file tree
Hide file tree
Showing 12 changed files with 280 additions and 114 deletions.
2 changes: 1 addition & 1 deletion .nvmrc
Original file line number Diff line number Diff line change
@@ -1 +1 @@
echo "20" > .nvmrc
lts/jod
21 changes: 21 additions & 0 deletions .yalc/detox-copilot/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2024 Wix Incubator

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
116 changes: 116 additions & 0 deletions .yalc/detox-copilot/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# Detox Copilot

A flexible plugin that drives your tests with human-written commands, enhanced by the power of large language models (LLMs).
While originally designed for Detox, Detox Copilot can be extended to **any other testing frameworks**.

It provides clear APIs to perform actions and assertions within your tests while interfacing with an LLM service to enhance the testing process.

## Quick Demo

Here's an example of how Copilot runs over a Detox test case:

<img src="copilot-demo.gif" width="800">

The test case is written in a human-readable format, and Copilot translates it into Detox actions on the fly.

**Not just Detox!** Copilot can be extended to any other testing frameworks.

## API Overview

High-level overview of the API that Detox Copilot exposes:

```typescript
/**
* Initializes the Copilot with the given configuration.
* Must be called before any other Copilot methods.
* @param config The configuration for the Copilot.
*/
init: (config: Config) => void;

/**
* Resets the Copilot instance.
* Must be called before each test to ensure a clean state (the Copilot uses the operations history as part of
* its context).
*/
reset: () => void;

/**
* Performs a testing operation or series of testing operations in the app based on the given `steps`.
* @returns The result of the operation(s), which can be a single value or an array of values for each step.
* @example Tap on the login button
* @example Scroll down to the 7th item in the Events list
* @example The welcome message should be visible
* @example The welcome message text should be "Hello, world!"
* @example [
* 'Tap on the login button',
* 'A login form should be visible',
* ]
*/
perform: (steps: string | string[]) => Promise<any | any[]>;
```

### Additional Note

In addition to the operations history, Copilot maintains a repository-level cache. If you need to ignore the current cache for any reason (e.g., when adding an action to the testing framework driver), you can set the environment variable `COPILOT_OVERRIDE_CACHE` to "true" before running your tests. This will ensure that the current cache is not taken into consideration and will override the existing one.

```shell
export COPILOT_OVERRIDE_CACHE=true
```

If you want to disable the override after setting it to "true" and revert to using the cache, you can set `COPILOT_OVERRIDE_CACHE` to "false"

```shell
export COPILOT_OVERRIDE_CACHE=false
```


## Integration with Testing Frameworks

Detox Copilot requires two main components to work:

### **Prompt Handler**

An adapter that interfaces with the LLM service to generate actions based on the provided prompts. For example, GPT, Gemini, Sonnet or any other LLM service.

#### `PromptHandler` Interface

```typescript
/**
* Sends a prompt to the AI service and returns the response.
* @param prompt The prompt to send to the AI service.
* @param image Optional path to the image to upload to the AI service that captures the current UI state.
* @returns The response from the AI service.
*/
runPrompt: (prompt: string, image?: string) => Promise<string>;

/**
* Checks if the AI service supports snapshot images for context.
*/
isSnapshotImageSupported: () => boolean;
```

### Testing Framework Driver

An adapter that interfaces with the testing framework to execute the generated actions. For example, Detox, Appium, Espresso, XCTest or any other testing framework.

In order for Copilot to work with the testing framework, the driver provides the API catalog and the JS context to execute the generated actions.

#### `TestingFrameworkDriver` Interface

```typescript
/**
* Takes a snapshot of the current screen and returns the path to the saved image.
* If the driver does not support image, return undefined.
*/
captureSnapshotImage: () => Promise<string | undefined>;

/**
* Returns the current view hierarchy in a string representation.
*/
captureViewHierarchyString: () => Promise<string>;

/**
* The available API methods of the testing framework.
*/
apiCatalog: TestingFrameworkAPICatalog;
```
35 changes: 35 additions & 0 deletions .yalc/detox-copilot/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{
"name": "detox-copilot",
"version": "0.0.24",
"description": "A flexible plugin that drives your tests with human-written commands, enhanced by the power of large language models (LLMs)",
"keywords": [
"detox",
"copilot",
"ai"
],
"author": "Asaf Korem <[email protected]>",
"homepage": "https://github.com/wix-incubator/detox-copilot",
"license": "MIT",
"main": "dist/index.js",
"types": "dist/index.d.ts",
"files": [
"dist"
],
"publishConfig": {
"registry": "https://registry.npmjs.org"
},
"repository": {
"type": "git",
"url": "git+https://github.com/wix-incubator/detox-copilot.git"
},
"scripts": {
"build": "tsc && tsc-alias",
"test": "jest",
"bump-version": "npm version patch && git push && git push --tags",
"release": "npm run test && npm run bump-version && npm run build && npm publish"
},
"bugs": {
"url": "https://github.com/wix-incubator/detox-copilot/issues"
},
"yalcSig": "bc9631e375f5cb89dcb536f9b7e9d552"
}
1 change: 1 addition & 0 deletions .yalc/detox-copilot/yalc.sig
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
bc9631e375f5cb89dcb536f9b7e9d552
33 changes: 32 additions & 1 deletion website/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions website/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
"@docusaurus/core": "3.6.1",
"@docusaurus/preset-classic": "3.6.1",
"@mdx-js/react": "^3.0.0",
"bootstrap": "^5.3.3",
"clsx": "^2.0.0",
"docusaurus-plugin-sass": "^0.2.5",
"prism-react-renderer": "^2.3.0",
Expand Down
1 change: 0 additions & 1 deletion website/src/components/Homepage/HomepageFeatures/index.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import clsx from 'clsx';
import Heading from '@theme/Heading';
import styles from './HomepageFeatures.module.scss';

const FeatureList = [
Expand Down
1 change: 0 additions & 1 deletion website/src/css/_root.scss
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
@use 'colors' as *;


:root {
@include define-detox-extra-colors();

Expand Down
15 changes: 6 additions & 9 deletions website/src/pages/index.js
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
import clsx from 'clsx';
import Link from '@docusaurus/Link';
import useDocusaurusContext from '@docusaurus/useDocusaurusContext';
import Layout from '@theme/Layout';
import HomepageFeatures from '@site/src/components/Homepage/HomepageFeatures';

import Heading from '@theme/Heading';
import CopilotGif from '/img/homepage/copilot-demo.gif';
import styles from './index.module.scss';

function HomepageHeader() {
Expand All @@ -15,17 +13,16 @@ function HomepageHeader() {
<h1 className={styles.title}>{siteConfig.title}</h1>
<h2 className={styles.tagline}>{siteConfig.tagline}</h2>
<div className={styles.buttons}>
<img
src={CopilotGif}
alt="Demo Gif"
className={styles.bottomRightGif}
/>
<Link
className="button button--secondary button--lg"
to="/docs/guides/integrating-with-testing-frameworks">
Getting started with Copilot
</Link>
{/* Add an img for the gif */}
<img
src="/img/homepage/copilot-demo.gif"
alt="Demo Gif"
className={styles.bottomRightGif}
/>
</div>
</div>
</header>
Expand Down
Loading

0 comments on commit a03e89d

Please sign in to comment.