We've started using Rush [1] at Buffer. Teams have been slowly migrating, and it has been great so far. We had (too many) repos with their own workflow and way too many different ways to build services, which has been annoying to maintain. I know teams use Rush at scale: Tiktok has ~450 projects, and Microsoft said they have ~700 projects.
To deploy the services locally, we use Tilt[2] (K8s for local). We want to be able to reproduce production as much as possible and remove developer overhead on how things work locally and in production.
Then come the issues with Docker and large node code bases:
1 challenge with large monorepos is the huge node_modules folder (rush among other tools put packages into a single large node_modules folder, and symlink every dependency there. It can contain millions of files and GBs, depending on how many 3rd-party libraries you use). On Linux, you can mount it without issues, but Mac has performance issues[3] with large folders.
We pre-build that huge node_modules into a "base" image, and each service in the monorepo pulls that base image and only mounts what's necessary (a few mb). So we can save that npm build time and don't need to copy all those files inside the containers. It is fine because package.json does not change that often. You need to do this pre-build locally then in your CI/CD -> dockerhub, so everyone can get it.
Another challenge is that you need to "watch" files to rebuild them. Watching all your files inside the monorepo isn't really viable. We use a dependency graph to know what services to watch and then copy the built files inside the Tilt containers.
This is where Bazel + rules_js ecosystem shines. First of all it lazily fetches dependencies which means if you are only building 1 out of X projects you are only getting your projects dependencies from npm, then it's only providing just those dependencies to the sandbox when you are doing node.js things like Typescript compilation. Finally you can assemble Docker/OCI images using rules_docker and js_image_layer which prevents the need for Dockerfiles at all, creates images with just exactly the dependencies needed (instead of the huge node_modules base layer which would need re-generating and downloading anytime a single dep changed) and better yet can be built pretty much instantly in most cases because it doesn't run Docker to do it.
To deploy the services locally, we use Tilt[2] (K8s for local). We want to be able to reproduce production as much as possible and remove developer overhead on how things work locally and in production.
Then come the issues with Docker and large node code bases:
1 challenge with large monorepos is the huge node_modules folder (rush among other tools put packages into a single large node_modules folder, and symlink every dependency there. It can contain millions of files and GBs, depending on how many 3rd-party libraries you use). On Linux, you can mount it without issues, but Mac has performance issues[3] with large folders.
We pre-build that huge node_modules into a "base" image, and each service in the monorepo pulls that base image and only mounts what's necessary (a few mb). So we can save that npm build time and don't need to copy all those files inside the containers. It is fine because package.json does not change that often. You need to do this pre-build locally then in your CI/CD -> dockerhub, so everyone can get it.
Another challenge is that you need to "watch" files to rebuild them. Watching all your files inside the monorepo isn't really viable. We use a dependency graph to know what services to watch and then copy the built files inside the Tilt containers.
Hope this can help people.
[1] https://rushjs.io
[2] https://tilt.dev/
[3] https://github.com/docker/roadmap/issues/7