Jean's Blog

一个专注软件测试开发技术的个人博客

0%

Midscene.js与Playwright集成

发表于 分类于
学习AI UI自动化测试学习笔记第二讲

在官方文档的介绍下，Midscene.js在做Web浏览器自动化可与Playwright和Puppeteer自动化测试框架集成，本章案例则是针对于集合Playwright进行实战

Playwright.js 是由微软开发的一个开源自动化库，主要用于对网络应用程序进行端到端测试（end-to-end test）和网页抓取。

环境说明：已安装Nodejs

第一步：初始化Playwright项目

  • 本地新建工作目录

    1
    2
    mkdir midscene-playwright
    cd midscene-playwright

    目录名根据自己需求创建即可

  • 执行初始化命令：npm init playwright@latest

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
      jinglv@localhost midscene-playwright % npm init playwright@latest
      
      > npx
      > create-playwright
      
      Getting started with writing end-to-end tests with Playwright:
      Initializing project in '.'
      ✔ Do you want to use TypeScript or JavaScript? · TypeScript
      ✔ Where to put your end-to-end tests? · tests
      ✔ Add a GitHub Actions workflow? (y/N) · false
      ✔ Install Playwright browsers (can be done manually via 'npx playwright install')? (Y/n) · true
      Initializing NPM project (npm init -y)…
      Wrote to /Users/jinglv/PycharmProjects/midscene-playwright/package.json:
      
      {
        "name": "midscene-playwright",
        "version": "1.0.0",
        "main": "index.js",
        "scripts": {
          "test": "echo \"Error: no test specified\" && exit 1"
        },
        "keywords": [],
        "author": "",
        "license": "ISC",
        "description": ""
      }
      
      
      
      Installing Playwright Test (npm install --save-dev @playwright/test)…
      
      added 3 packages in 1s
      Installing Types (npm install --save-dev @types/node)…
      
      added 3 packages in 640ms
      Writing playwright.config.ts.
      Writing tests/example.spec.ts.
      Writing tests-examples/demo-todo-app.spec.ts.
      Writing package.json.
      Downloading browsers (npx playwright install)…

    ​	一直回车即可	

    注意问题：

    在初始工程时，执行到`Downloading browsers (npx playwright install)…`卡在这里不动了，也没有任何报错，经过尝试发现是权限问题，在初始化命令加上sudo即可，`sudo npm init playwright@latest`

    Mac电脑是这样，没有Windows电脑，如果Windows遇到同样问题，在查找其他资料查看

    # 第二步：查看项目目录

    完成项目初始化后，查看下项目目录结构

    ```shell
    ├── e2e
    │   └── example.spec.ts
    ├── package-lock.json
    ├── package.json
    ├── playwright.config.ts
    ├── tests
    │   └── example.spec.ts
    └── tests-examples
        └── demo-todo-app.spec.ts

项目根目录文件

  • playwright.config.ts - Playwright 主配置文件，定义了测试目录、浏览器项目、并行设置等
  • package.json -项目依赖管理，包含
  • package-lock.json- 锁定依赖版本
  • .gitignore - Git 忽略文件，包含 Playwright 相关的输出目录

测试目录

  • tests/ - 主要测试目录（在 playwright.config.ts 中配置为 testDir）
    • example.spec.ts - 基础示例测试，包含标题检查和导航测试
  • tests-examples/ - 详细示例测试目录
  • demo-todo-app.spec.ts - 完整的 TodoMVC 应用测试套件，展示各种测试模式
  • e2e/ - 另一个测试目录
    • example.spec.ts - 与 tests/ 目录中相同的基础测试

自动生成的输出目录（被 .gitignore 忽略）

  • 配置了三个浏览器项目：Chromium、Firefox、WebKit
  • 启用了完全并行测试 (fullyParallel: true)
  • 在 CI 环境中有特殊配置（重试次数、工作进程数）
  • 使用 HTML 报告器作为默认输出格式

第三步：配置.env环境变量

在项目目录下，新建文件为.env，将模型配置的内容，写入到该文件中去，如配置通义千问的模型：

1
2
3
4
5
OPENAI_API_KEY="sk-xxxxxxxxxxxxxxxxx"
OPENAI_BASE_URL="https://dashscope.aliyuncs.com/compatible-mode/v1"
MIDSCENE_MODEL_NAME="qwen-vl-max-latest"
MIDSCENE_USE_QWEN_VL=1

第四步：新增依赖

1
npm install @midscene/web --save-dev

第五步：修改playwright.config.ts内容

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
import { defineConfig, devices } from '@playwright/test';

/**
 * Read environment variables from file.
 * https://github.com/motdotla/dotenv
 */
// import dotenv from 'dotenv';
// import path from 'path';
// dotenv.config({ path: path.resolve(__dirname, '.env') });

/**
 * See https://playwright.dev/docs/test-configuration.
 */
export default defineConfig({
  testDir: './e2e',
  timeout: 900 * 1000,
  /* Run tests in files in parallel */
  fullyParallel: true,
  /* Fail the build on CI if you accidentally left test.only in the source code. */
  forbidOnly: !!process.env.CI,
  /* Retry on CI only */
  retries: process.env.CI ? 2 : 0,
  /* Opt out of parallel tests on CI. */
  workers: process.env.CI ? 1 : undefined,
  /* Reporter to use. See https://playwright.dev/docs/test-reporters */
  reporter: [["list"], ["@midscene/web/playwright-report"]],
  /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
  use: {
    /* Base URL to use in actions like `await page.goto('/')`. */
    // baseURL: 'http://127.0.0.1:3000',

    /* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
    trace: 'on-first-retry',
  },

  /* Configure projects for major browsers */
  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },

    // {
    //   name: 'firefox',
    //   use: { ...devices['Desktop Firefox'] },
    // },

    // {
    //   name: 'webkit',
    //   use: { ...devices['Desktop Safari'] },
    // },

    /* Test against mobile viewports. */
    // {
    //   name: 'Mobile Chrome',
    //   use: { ...devices['Pixel 5'] },
    // },
    // {
    //   name: 'Mobile Safari',
    //   use: { ...devices['iPhone 12'] },
    // },

    /* Test against branded browsers. */
    // {
    //   name: 'Microsoft Edge',
    //   use: { ...devices['Desktop Edge'], channel: 'msedge' },
    // },
    // {
    //   name: 'Google Chrome',
    //   use: { ...devices['Desktop Chrome'], channel: 'chrome' },
    // },
  ],

  /* Run your local dev server before starting the tests */
  // webServer: {
  //   command: 'npm run start',
  //   url: 'http://127.0.0.1:3000',
  //   reuseExistingServer: !process.env.CI,
  // },
});

主要修改内容的注意点为：

  • 测试脚本的目录为：testDir: ‘./e2e’，将测试脚本都放在这个目录下
  • 生成报告的格式为：reporter: [[“list”], [“@midscene/web/playwright-report”]]

第六步：扩展test实例

在目录e2e下新建文件fixture.ts，写入以下内容

1
2
3
4
5
6
7
import { test as base } from '@playwright/test';
import type { PlayWrightAiFixtureType } from '@midscene/web/playwright';
import { PlaywrightAiFixture } from '@midscene/web/playwright';
import 'dotenv/config';

export const test = base.extend<PlayWrightAiFixtureType>(PlaywrightAiFixture());

第七步：编写一个测试用例

用例编写，API参考：https://midscenejs.com/zh/API.html

在e2e目录下，新建文件为：ebay-search.spec.ts，官方示例代码如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
import { expect } from '@playwright/test';
import { test } from './fixture';

test.beforeEach(async ({ page }) => {
  page.setViewportSize({ width: 400, height: 905 });
  await page.goto('https://www.ebay.com');
  await page.waitForLoadState('networkidle');
});

test('search headphone on ebay', async ({
  ai,
  aiQuery,
  aiAssert,
  aiInput,
  aiTap,
  aiScroll,
  aiWaitFor,
}) => {
  // 使用 aiInput 输入搜索关键词
  await aiInput('Headphones', '搜索框');

  // 使用 aiTap 点击搜索按钮
  await aiTap('搜索按钮');

  // 等待搜索结果加载
  await aiWaitFor('搜索结果列表已加载', { timeoutMs: 5000 });

  // 使用 aiScroll 滚动到页面底部
  await aiScroll(
    {
      direction: 'down',
      scrollType: 'untilBottom',
    },
    '搜索结果列表',
  );

  // 使用 aiQuery 获取商品信息
  const items =
    await aiQuery<Array<{ title: string; price: number }>>(
      '获取搜索结果中的商品标题和价格',
    );

  console.log('headphones in stock', items);
  expect(items?.length).toBeGreaterThan(0);

  // 使用 aiAssert 验证筛选功能
  await aiAssert('界面左侧有类目筛选功能');
});

注意：测试代码文件命名，必须以.spec.ts结尾

执行命令：npx playwright test ./e2e/ebay-search.spec.ts，这个命令是无头模式，要想有头模式，在命令上增加—headed即可

第八步：查看测试报告

执行完成后，项目目录下，会多出一个midscene_run 目录，进入到该目录，目录下有一个report目录，进入该目录可以看到已经执行的报告html

image-20250716150744691