Framework prototype

[filipesmedeiros]

Purpose

To showcase how the codebase might look if adopting a full-stack React framework.
Show some code patterns, conventions, advantages and trade-offs.

Framework chosen

Next.js was chosen as the prototype framework, due to its maturity and given it's recommended officially by the React team. Note that Create React App has been deprecated.

Architecture overview

With a framework like this, we get one "package" with both serving web pages and the API. Of course this is not a necessity per se, but given the simplicity of the project, a dedicated package for the API does not seem beneficial. I will give examples of this further down.

One server, and one piece of the project, will be responsible for both implementing server-side routing logic (when to serve each page/component), client-side transitions (between pages), API routes (like saving new Simlin projects) and rendering client React code, all simplified under one workflow.

As a key concept: all web routes are coded wither as React components! For API routes, you write the usual Javascript functions (similar to Express).

Pieces not covered by this framework

Both the database and authentication are not covered by default by the framework, and have to be added on top. This has been included as an example, although simplified.

Examples

How to implementing simple server-side routing logic

Routing both server side and client side (important!) is done on a file-system basis. This means that the folder structure of the project determines which routes are available on the web server.

For example, you can see that, inside the app directory (the root of the routing), there is a page.tsx file. This will serve the / path. On the other hand, we also have app/login/page.tsx, which will serve the /login path.

For now, we can ignore the (home) folder, since it is a Next.js specific convention, that we can explore in the future (docs here).

How to implementing simple client-side routing logic

Like said above, the file-system routing applies to both client and server routing. Next.js (and others) take care of this by themselves. For example, Next.js will serve the / page from the server, but if I navigate to /page2, the new React components will be streamed to the client as needed, and replaced by React itself. You can see an example of this working between pages / (home page) and /new (new project page). The header is not being rerendered, only the "body" of the page.

How to implement authentication

Authentication uses Firebase, like before, by taking advantage of its most recent updates, such as initializeServerApp (blog) to merge well with a full-stack framework.

How to authenticate a route

Authenticating routes (forbidding or redirecting) is very easy. Simply check for authentication, in the route itself (which is a React component!):

https://github.com/filipesmedeiros/simlin/blob/architecture-tests/src/nextjs/src/app/(home)/layout.tsx#L11

If the request is not authenticated, redirect it:

https://github.com/filipesmedeiros/simlin/blob/architecture-tests/src/nextjs/src/app/(home)/layout.tsx#L13

Else (the request is authenticated), return the normal React components:

https://github.com/filipesmedeiros/simlin/blob/architecture-tests/src/nextjs/src/app/(home)/layout.tsx#L16-L19

How to fetch data from the database

This is a trick question, as it can be done in two ways. Either client-side (normal browser fetch), or as part of the render process itself. For this the framework runs the code server-side to build the proper HTML and React structure and send the JIT-built HTML to the client, with the proper data included.

For example, to render the projects page (the home page), I simply retrieve the projects from the database (after getting the authenticated user):

https://github.com/filipesmedeiros/simlin/blob/architecture-tests/src/nextjs/src/app/(home)/page.tsx#L8-L10

After that, I simply have to return the React component, with the project data used as needed:

https://github.com/filipesmedeiros/simlin/blob/architecture-tests/src/nextjs/src/app/(home)/page.tsx#L12-L29

How this happens is up to the framework, but essentially it is rendering the React code server side and sending the HTML, while at the same time sending the needed JS to the client to enable client-side transitions.

From what I can understand, this removes a much of the need for Protobufs, but I might be wrong!

But what about client-side data fetches?

Those can, of course, exist as well. In fact, if we need them, they are just the same as before: useEffect(() => {fetchData}, []) (kind of equivalent to componentDidMount).

How to do data mutations (form submissions, for example)

Doing data mutations (creates, update, etc) can be done in various ways. The most obvious for the prototyped feature is using forms (native HTML forms). Next.js supports this out of the box:

https://github.com/filipesmedeiros/simlin/blob/architecture-tests/src/nextjs/src/app/(home)/new/NewProjectForm.tsx#L39

We can use useActionState in a client component, in this case only to allow for error handling (showing error reasons to the person, if the project creation fails):

https://github.com/filipesmedeiros/simlin/blob/architecture-tests/src/nextjs/src/app/(home)/new/NewProjectForm.tsx#L36

We use a React Server Action to process the form submission, after some inicial client-side processing:

https://github.com/filipesmedeiros/simlin/blob/architecture-tests/src/nextjs/src/lib/createProjectAction.ts

For now, model conversion is done client side. This can be easily changed in the future, though!

Tooling

In terms of tooling, our life is also simplified, because we have one less package to worry about, and client/server are essentially both the same. I believe much of the complexity of handling Typescript config, React versions, Node.js versions, etc. can be delegated to the framework.

Conclusion

Please let me know if you have any questions or suggestions :)

[bpowers]

Thanks for getting this up so quick - at a high level I think this makes sense. I'd like to keep the client side data loading and keep the server ~ as simple and close to what it is (the changes to a directory structure that defines the routing make sense , but I'd like to start with eg assets compiled once and served rather than server side rendering). I didn't look at the details yet - I will try to carve some time out to do that later today or tomorrow!

[bpowers]

@filipesmedeiros one other thing to confirm - the Editor component and all the drawing/ and other components under it right now take care to not eg do network calls or depend on the specific server integration. The idea is that this would all be handled by a wrapper (I think HostedWebEditor) that makes it simple to embed the stateless + agnostic Editor in other things, like a blog or online textbook/tutorial. It would be great if we could keep the next.js integration to the same footprint. Will that be possible? Let me know if I'm being unclear or should explain further (currently on phone)

[filipesmedeiros]

Hey @bpowers! As for the Editor issue, I don't think that's a problem at all. We simply create an "embedded" route that serves a static and non-authenticated editor. Like you said, this should work out of the box!

As for the server/client thing, I think we should clarify this a bit: what exactly is the concern with having server side rendering? Are you trying to avoid some issue? I ask this to try to meet your demands and expectations, and to consider all possible options. Of course it is possible to make the entire frontend static and just make client-side data fetching, but why is that your preferred method? We always have to have a spinning server for the API (unless we rely totally on Firebase to be the only server running, which is also a possibility, technically)

To clarify myself further: a full-stack framework is meant, for our use case, to replace both the client and the server (as, with the current prototype, it itself is a Node.js server).

[bpowers]

Hey @bpowers! As for the Editor issue, I don't think that's a problem at all. We simply create an "embedded" route that serves a static and non-authenticated editor. Like you said, this should work out of the box!

I think I was more asking/confirming: can <Editor> and the react compoonents the editor use stay next.js oblivious? looking at the diff, they seem untouched so far. my goal/desire here is to keep them usable in other contexts in the future

As for the server/client thing, I think we should clarify this a bit: what exactly is the concern with having server side rendering? Are you trying to avoid some issue? I ask this to try to meet your demands and expectations, and to consider all possible options. Of course it is possible to make the entire frontend static and just make client-side data fetching, but why is that your preferred method? We always have to have a spinning server for the API (unless we rely totally on Firebase to be the only server running, which is also a possibility, technically)

To clarify myself further: a full-stack framework is meant, for our use case, to replace both the client and the server (as, with the current prototype, it itself is a Node.js server).

it probably doesn't matter much, but I see server-side rendering as complexity we don't need. Generally if things can be simpler, I prefer it. The biggest compelling argument I can come up with is that it could be useful for search indexing to server-side render public models

[filipesmedeiros]

I think I was more asking/confirming: can <Editor> and the react compoonents the editor use stay next.js oblivious? looking at the diff, they seem untouched so far. my goal/desire here is to keep them usable in other contexts in the future

Yes :) I think that's possible and also the most natural course of action!

it probably doesn't matter much, but I see server-side rendering as complexity we don't need. Generally if things can be simpler, I prefer it. The biggest compelling argument I can come up with is that it could be useful for search indexing to server-side render public models

It's funny because I see it the exact opposite! For me, SSR is closer to static. I can more easily think "One browser route, one response". Client side rendering can increase the round trips a lot. For example, if a person who is logged in goes to /, they will get an HTML skeleton and JS, then React has to load and fetch the projects, and then render those project in the skeleton. If we have access to everything from the start, why not just send the final HTML to the browser? The data-passing logic becomes much simpler, in my view.

Of course the actually interactive parts of the application (in our case, it's just the editor, at least for the foreseeable future) are client-heavy, and they will stay that way. In essence, by using a framework we are basically: saving network roundtrips and simplifying data management, at least from my point of view!

In any case, this is your decision to make :)

If you don't like the direction where this takes the codebase and development experience (the end-user experience should stay almost exactly the same), we can keep the current architecture!

[bpowers]

ok, thanks for sharing! I still feel likely to accept this, even if we different things are simple to us. From my perspective, next.js is a big pile of code, and now sometimes code runs client side, sometimes it runs server side, and that feels like more cognitive load to keep in mind when writing code compared to "this pile runs in browser" and "this much smaller pile runs on server".

again: not trying to dampen your enthusiasm, and eager to see what this ends up looking like in full

[filipesmedeiros]

Hey Bobby!

Now that you put it like that:

next.js is a big pile of code, and now sometimes code runs client side, sometimes it runs server side, and that feels like more cognitive load to keep in mind when writing code compared to "this pile runs in browser" and "this much smaller pile runs on server".

I totally understand your side! I think it comes down to something like this: Next.js promises to take care of the boring side of client/server coordination, and asks for your trust. But for that to work, from a developer point of view, it has to feel like you're doing less work, it has to feel ergonomic and natural to code this way. I can totally see how this would not be the preferred way to code a web application, even if I "grew up" with Next.js and it feels natural to me.

I will port the entire application and we can compare side-by-side with the finalised version (there isn't a lot left, to be honest!)