One moment…
Building the next page.
One moment…
Building the next page.
MERN + Socket.io
A scalable multi-tenant activity feed system built with the MERN stack, featuring cursor-based pagination, real-time updates, infinite loading, optimistic UI updates and a scalable architecture for large activity datasets.
ScaleStreamOptimizeRepeat!
Idea to ImpactOne Sprint at a Time
This project is a multi-tenant activity feed system built using the MERN stack with a focus on scalability, performance and real-time user experiences.
The system simulates the type of activity feed experience found in platforms such as Instagram and LinkedIn, where users continuously load activity data while new events can arrive in real time.
The project focuses on several important engineering challenges:
The goal was to design an activity feed that remains efficient as the amount of activity data grows.
Instead of relying on traditional offset or skip-based pagination, the system uses cursor-based pagination to efficiently navigate through large datasets.
Real-time updates are handled using Socket.io, while React Query manages server state and infinite loading on the frontend.
The system is designed around a multi-tenant model.
Activity records are associated with a specific tenantId, allowing the same application architecture to serve multiple tenants while keeping activity queries scoped to the appropriate tenant.
This tenant identifier also becomes an important part of the database indexing strategy.
Traditional skip-based pagination becomes increasingly inefficient as datasets grow because MongoDB must scan and discard the documents represented by the skipped records.
The system therefore uses cursor-based pagination.
The feed queries activities using a cursor based on createdAt:
db.activities.find({
tenantId,
createdAt: { $lt: cursorDate }
})
.sort({ createdAt: -1 })
.limit(20);
This allows MongoDB to perform range-based scanning rather than repeatedly skipping large numbers of documents.
The pagination query requires an index that matches the tenant filter and sorting strategy.
The project uses:
{
tenantId: 1,
createdAt: -1
}
This allows MongoDB to efficiently filter activities by tenant while maintaining the required ordering for cursor-based pagination.
The frontend uses React Query with infinite loading to progressively retrieve activity pages.
Instead of loading the entire feed at once, additional activities are requested as the user continues scrolling.
This reduces the amount of data loaded initially while providing a continuous feed experience.
Socket.io is used to deliver new activities to connected clients in real time.
When a new activity is created, connected clients can receive the event without waiting for the next manual refresh.
This creates a more dynamic experience where the activity feed can update while the user is viewing it.
The system also implements optimistic UI updates.
When a new activity is created, the frontend does not wait for the server response before updating the interface.
Instead:
This makes interactions feel immediate while maintaining consistency with the backend.
Optimistic updates introduce the possibility that the server request can fail after the UI has already changed.
The rollback strategy therefore relies on React Query invalidation and refetching.
When the API request fails:
This avoids leaving stale optimistic data in the feed.
A major part of the project was understanding how the feed behaves as the dataset grows.
The system considers several production performance metrics:
These metrics provide visibility into both database and application performance.
The architecture also considers how the system could scale to approximately 50 million activities per tenant.
Scaling requires looking beyond a single database query and considering multiple layers of the system.
The project explores:
The purpose is to understand how an initially simple activity feed architecture can evolve as traffic and data volume increase.
A multi-tenant system can encounter situations where one tenant generates significantly more traffic or activity than others.
This creates a potential hot tenant problem where a disproportionate amount of traffic or data is concentrated around a single tenant.
The scaling strategy therefore needs to account for tenant-level traffic distribution rather than assuming every tenant behaves identically.
Large activity datasets continuously grow over time.
A retention strategy becomes important when historical activity is no longer required in the primary feed.
Data retention can help control storage growth and keep the active dataset manageable as the number of activities increases.
Real-time delivery introduces another scaling dimension.
As the number of connected clients increases, Socket.io delivery needs to be considered alongside API and database scaling.
The real-time layer therefore needs to be designed independently enough that increasing feed traffic does not unnecessarily overload the database or application layer.
This project was an exercise in understanding that building a feature is only one part of engineering.
An activity feed can initially look simple:
fetch activities → display activities
But once the system needs to support multiple tenants, large datasets, infinite scrolling, real-time events and high traffic, the underlying architecture becomes much more important.
The project helped explore how database indexes, pagination strategy, caching, real-time communication and frontend state management work together as one system.
The most valuable part of this project was thinking beyond the initial implementation.
Questions such as:
turn a simple CRUD application into a real system-design problem.
This project was built to explore those questions through an actual working implementation.