Track: HTTP client support on React Native for Android and iOS

[acostalima]

HTTP client - January 18-29 ⌛

• Uncover and document issues related to HTTP client's operation in React Native
• Create app to showcase HTTP client working in React Native
  • https://github.com/ipfs-shipyard/react-native-ipfs-demo
• Create package to polyfill and patch missing or partially supported web and core APIs
  • https://github.com/acostalima/react-native-polyfill-globals, in use by the demo app
  • Track: Error: Cannot create URL for blob charpeni/react-native-url-polyfill#223
• Propose fix for HTTP client factory's URL normalization logic
  • Track: HTTP client factory: Invalid URL #3331
• Fix default base URL as per URL spec
  • Track: fix: fix default base as per URL spec hugomrdias/iso-url#133
• Patch HTTP client's pubsub.subscribe method to pass an option to fetch to activate text streaming support (custom option for React Native)
  • Track: Enable text streaming support for React Native (iOS and Android) on pubsub.subscribe #3488 (closed)
• Have HTTP client's test suite running on React Native (with @hugomrdias' support 🙏)
  • Track: Add better guard against location definition hugomrdias/iso-url#154
  • Track: Add React Native (iOS and Android) support js-ipfs-utils#91
  • add RN iOS support to aegir
  • Track: Adapt HTTP client tests for React Native (iOS and Android) #3517

React Native Test Runner - January 4-8 ✅

A CLI tool in line of what @hugomrdias has done with playright-test but for React Native. The first version includes support for Zora test runner.

Repo: https://github.com/acostalima/react-native-test-runner

• Concept validation: use React Native's bundler, Metro, to report test results from an iOS or Android app to a Node process.
  • Uses Metro and React Native's CLI APIs
• Find iOS simulator by name and OS version
• Start/stop headless iOS simulador
• Install/launch iOS app on simulator
  • Tools: xcrun simctl, Wix's AppleSimulatorUtils, ...
• Build iOS app in debug mode programatically
  • Tools: xcodebuild, pod
  • Output binary location: ??
• Metro bundler: server.close() hangs after an app connects to the server and then when it's terminated
  • Track: Server never closes when using Metro API facebook/metro#614
• Fix Zora's exports in package.json for React Native
  • Track: Add package.json to exports for React Native lorenzofox3/zora#74
• Find Android emulator by name and OS version
• Build Android app in debug mode programatically
  • Tools: gradlew
  • Output binary location: ??
• Install/launch Android app on emulator
  • Tools: adb, emulator, avdmanager
• Start/stop headless Android emulator
• CLI
  • Tools: meow, yarn
• Tests
• Add README
• Setup CI pipeline in GitHub Actions
  • Android job
  • iOS job
  • Android cache
  • iOS cache
  • npm cache
• Use react-native init to initialize a new React Native app instead and distributing it
  • When the test runner is installed as dev dependency, React Native itself and its dependencies may be hoisted
  • There are files in the Android and iOS projects that expect React Native, its CLI and Metro to be available at the top level node_modules directory by hardcoded relative path look ups (e.g. ../node_modules/react-native). As such, the Android and iOS projects cannot be packaged and released to npm.
  • Accept a CLI option to pass React Native semver to use
  • Create/delete RN test app directory
  • Make RN and Metro requires relative to RN's test app directory
  • Use default package name and bundle ID defined by react-native init for Android and iOS respectively to launch the apps
  • Check whether Babel config file needs to be copied to the RN test app's directory

References & Inspiration

• https://github.com/react-native-community/cli/tree/master/packages/platform-android/src
• https://github.com/react-native-community/cli/tree/master/packages/platform-ios/src
• https://github.com/react-native-community/cli/blob/master/packages/cli/src/commands/start/runServer.ts
• https://github.com/react-native-community/cli/blob/master/packages/cli/src/commands/start/watchMode.ts
• https://facebook.github.io/metro/docs/getting-started
• https://facebook.github.io/metro/docs/configuration
• https://github.com/facebook/metro/blob/b2528d40c4/packages/metro/src/HmrServer.js
• https://github.com/facebook/metro/blob/b2528d40c4/packages/metro/src/lib/attachWebsocketServer.js
• https://github.com/facebook/metro/blob/b2528d40c4/packages/metro/src/index.js
• https://github.com/react-native-community/cli/blob/master/packages/cli/src/tools/loadMetroConfig.ts
• https://github.com/react-native-community/cli/tree/master/packages/cli-server-api/src
• https://github.com/react-native-community/cli/blob/c09733ad85ec79b6f0475cd5b0237a6fa20bbaac/packages/cli/src/tools/metroPlatformResolver.ts
• https://github.com/wix/Detox/blob/master/docs/Guide.RunningOnCI.md
• https://github.com/wix/Detox/blob/master/docs/Introduction.AndroidDevEnv.md
• https://github.com/wix/Detox/blob/master/docs/Introduction.IosDevEnv.md

Fetch API - January 8-15 ✅

Technical discussion: react-native-community/fetch#4

• Implement Fetch API with ReadableStream support and text streaming built in top of React Native's networking API
  • https://github.com/react-native-community/fetch
• Report and fix an issue about async iteration of ReadableStream
  • Unhandled promise rejection with TypeError when using async iterator MattiasBuelens/web-streams-polyfill#68
• Take existing tests from GitHub's Fetch implementation and make them run in a React Native app
• Add tests for text streaming support (chunked encoding transfers)
• Setup CI pipeline in GitHub Actions to run Fetch API's tests automatically
• Update README
• Publish v1.0.0
• Update https://github.com/acostalima/react-native-polyfill-globals to setup https://github.com/react-native-community/fetch
• Add @react-native-community/[email protected] as a dependency to https://github.com/ipfs-shipyard/react-native-ipfs-demo
• Update https://github.com/ipfs-shipyard/react-native-ipfs-demo README

React Native - January 18-22 ⌛

• Usage of console.error and console.warn when consuming a Blob with FileReader.readAsDataURL.
  • Track: Native error is thrown when blob is read with FileReader.readAsDataURL facebook/react-native#30378
• Land PR to implement FormData.set method
  • Patch in place: https://github.com/acostalima/react-native-polyfill-globals/blob/a59ebe892fe8f05991ec7e119f21b49a3e4b500d/patches/react-native%2B0.63.3.patch#L86
  • Track: Implement FormData.set method and handle Blob instances facebook/react-native#30767
• Land PR to make FormData.getParts capable of handling Blob instances
  • Patch in place: https://github.com/acostalima/react-native-polyfill-globals/blob/a59ebe892fe8f05991ec7e119f21b49a3e4b500d/patches/react-native%2B0.63.3.patch#L108
  • Track: Implement FormData.set method and handle Blob instances facebook/react-native#30767
• Land PR to implement FileReader.readAsArrayBuffer
  • Patch in place: https://github.com/acostalima/react-native-polyfill-globals/blob/a59ebe892fe8f05991ec7e119f21b49a3e4b500d/patches/react-native%2B0.63.3.patch#L19
  • Track: Implement FileReader.readAsArrayBuffer facebook/react-native#30769

Related work

Jet by Invertase

Jet lets you bring your React Native JS code into NodeJS and test it mock free and native testing code free - ideal for testing React Native modules e2e.

The developers behind React Native Firebase. Currently, the project is a PoC and does not have a dedicated GitHub repo.

Blob download/upload

• React Native Blob Courier: https://github.com/edeckers/react-native-blob-courier
• React Native Fetch Blob: https://github.com/joltup/rn-fetch-blob/
  • Currently not maintained: Unmaintained joltup/rn-fetch-blob#666
• Expo File System: https://docs.expo.io/versions/latest/sdk/filesystem/

[acostalima]

Having tests running in React Native will be tricker than I initially expected. I had a look at Detox a couple of weeks ago and I was under the impression it allowed to run any kind of tests in React Native's environment. However, I had a deeper look most recently and Detox's architecture and inner workings and it turns out it's not possible at all.

Detox supports test runners such as Mocha and Jest, but they actually run in a separate Node process, external to the app's runtime. Detox's assertions and matchers communicate with the app back and forth through a web socket and such APIs are designed exclusively for UI testing. Therefore, we will not be able to use Detox to run Fetch's tests.

I was not able to find anything else out there in open source that achieves what we're looking for, i.e. having tests actually running in a React Native app just like Playwright does so in the web counterpart. Thus, we have to come up with something of our own.

The first step is to explore how to have Mocha running in React Native (eventually discuss this with Mocha's team as well). I did have a quick shot at it yesterday, but I got a bunch of errors so I'll have to go in deeper. If Mocha reveals to be quite problematic in this regard, I can try with something more simpler, such as Tape.

The second step is to design a solution to extract test results out of the app. A HTTP server should do, since we only need unidirectional communication where the app is the client. Once the JS kicks in upon app startup, the test runner would start immediately.

The third and last step is to orchestrate the whole thing with a Node process: build the app, start te app, setup the server, signal the server that the tests are about to run, wait for the tests to run to completion, send the results and report them.

[hugomrdias]

try https://github.com/lukeed/uvu or https://github.com/lorenzofox3/zora these should be simpler to integrate in rn and wrap with something similar to playrwright-test for rn.

[acostalima]

@hugomrdias thanks! 💪

[acostalima]

@autonome there has been a minor setback with regards to the developments of the test runner for RN. I had a discussion with @hugomrdias today, and we concluded that some changes are necessary to make it work properly when using the runner as a dependency (details above). It will require a bit more work but it's nothing to be concerned about. 😉

[hugomrdias]

@autonome when @acostalima updates the OP you probably don't get a notification, we can setup a different process to make sure you get a ping about updates.

[jacobheun]

Adding #3076 for tracking as well

[acostalima]

Finishing up the changes me and @hugomrdias agreed on in order to avoid packaging and distributing the test app to npm: acostalima/react-native-test-runner#2. The tests are passing locally, but failing on CI at the moment. I'm currently fixing this. Fixed!

[acostalima]

This week:

• Finish fetch's test suite, update README and publish v1.0.0.
• Squeeze in a couple of PRs to React Native.

[acostalima]

The Fetch implementation for React Native is ready for v1.0.0 release. Meanwhile, I've asked Christoph from Facebook to give me permissions to publish the package in react-native-community scope.

CC @autonome

[acostalima]

Last week

• Updated demo app repo, in particular the contents of the README to better explain how to properly setup an app to use the HTTP client for the time being given the known limitations and issues.
• Concluded fetch's test suite and updated the README. The publication of v1.0.0 remains pending until I am granted permission to publish the package under react-native-community's scope.
• Updated polyfill globals repo to remove a few no longer necessary patches and add the setup to polyfill React Native's environment with the alternative fetch implementation.
• Started to work on the changes to submit to React Native's repo.

This week

• Submit PRs to React Native, namely:
  • Implement .set method in FormData.
  • Make FormData.getParts capable of handling Blobs.
  • Implement FileReader.readAsArrayBuffer.
• Submit PR to HTTP client to enable text streaming on fetch for React Native via custom option.
• Investigate whether/how mocha can be used in React Native's runtime.

CC @autonome

[acostalima]

Last week

• Landed PRs to React Native:
  • Implement FormData.set method and handle Blob instances facebook/react-native#30767
  • Implement FileReader.readAsArrayBuffer facebook/react-native#30769
• Landed PRs to JS IPFS:
  • Enable text streaming support for React Native (iOS and Android) on pubsub.subscribe #3488
• Added support for Mocha in React Native Test Runner.

This week

• Published [email protected] 🎉
• Working together with @hugomrdias to get HTTP client's test suite going in React Native.

CC @autonome

[acostalima]

Last week

• PRs to complete the integration of the HTTP client runtime and test suite in React Native are ready for peer review.

Next

• Waiting for peer review and support to complete the work on IPFS end @hugomrdias.
• Waiting for peer review on PRs submitted to React Native.

CC @autonome

[rikur]

This looks very promising, thank you @acostalima! As it may take a while to get any feedback from RN team, I'd like to ask if this is currently usable with the polyfills/patches?

[acostalima]

@rikur yes, you should be able to use most methods of the HTTP client with the available polyfills and patches. Have a look at https://github.com/ipfs-shipyard/react-native-ipfs-demo to see how to make it work.

Note that I'm not sure whether the patch available at https://github.com/acostalima/react-native-polyfill-globals/blob/master/patches/react-native%2B0.63.3.patch can be applied to RN versions in the range <0.63.3 >=0.64.0. I know for sure the patch is compatible with 0.63.3 and 0.63.4.

If you run into any issues, please let me know. 💪

[Haianh9999]

@BigLep
Currently ipfs-http-client can work properly with react native yet?. I tried the latest version and got the error 🙇

[SgtPooki]

js-ipfs is being deprecated in favor of Helia. You can #4336 and read the migration guide.

Please feel to reopen with any comments by 2023-06-02. We will do a final pass on reopened issues afterward (see #4336).

@Haianh9999 note that I'm pushing for getting an example using react-native in Helia. I posted a comment at ipfs/helia#43 (comment), please feel free to contribute an example if you get time!