extern/mediacms
1
0
Fork 0
mirror of https://github.com/mediacms-io/mediacms.git synced 2024-11-22 08:13:33 +01:00
Code Issues Packages Projects Releases Wiki Activity
ba94989e6a
mediacms/docs/developers_docs.md
Markos Gogoulos c27e3caff6
documentation improvements (#261)
2021-08-01 19:47:35 +03:00

4.7 KiB
Raw Blame History

Developers documentation

Table of contents

How to contribute

Before you send a PR, make sure your code is properly formatted. For that, use pre-commit install to install a pre-commit hook and run pre-commit run --all and fix everything before you commit. This pre-commit will check for your code lint everytime you commit a code.

Checkout the Code of conduct page if you want to contribute to this repository

System architecture

API documentation

API is documented using Swagger - checkout ot http://your_installation/swagger - example https://demo.mediacms.io/swagger/ This page allows you to login to perform authenticated actions - it will also use your session if logged in.

Working with Docker tips

To perform the Docker installation, follow instructions to install Docker + Docker compose (docs/Docker_Compose.md) and then build/start docker-compose-dev.yaml . This will run the frontend application on port 8088 on top of all other containers (including the Django web application on port 80)

docker-compose -f docker-compose-dev.yaml build
docker-compose -f docker-compose-dev.yaml up

Frontend application changes

Eg change frontend/src/static/js/pages/HomePage.tsx , dev application refreshes in a number of seconds (hot reloading) and I see the changes, once I'm happy I can run

docker-compose -f docker-compose-dev.yaml -T frontend npm run dist

And then in order for the changes to be visible on the application while served through nginx,

cp -r frontend/dist/static/* static/

POST calls: cannot be performed through the dev server, you have to make through the normal application (port 80) and then see changes on the dev application on port 8088. Make sure the urls are set on frontend/.env if different than localhost

Media page: need to upload content through the main application (nginx/port 80), and then use an id for page media.html, for example http://localhost:8088/media.html?m=nc9rotyWP

There are some issues with CORS too to resolve, in order for some pages to function, eg the manage comments page

Backend application changes

After I make changes to the django application (eg make a change on files/forms.py) in order to see the changes I have to restart the web container

docker-compose -f docker-compose-dev.yaml restart web

How video is transcoded

Original files get uploaded to the application server, and they get stored there as FileFields.

If files are videos and the duration is greater than a number (defined on settings, I think 4minutes), they are also broken in chunks, so one Encode object per chunk, for all enabled EncodeProfiles.

Then the workers start picking Encode objects and they transcode the chunks, so if a chunk gets transcoded correctly, the original file (the small chunk) gets replaced by the transcoded file, and the Encode object status is marked as 'success'.

original.mp4 (1G, 720px)--> Encode1 (100MB, 240px, chunk=True), Encode2 (100MB, 240px, chunk=True)...EncodeXX (100MB, 720px, chunk=True) ---> when all Encode objects are success, for a resolution, they get concatenated to the original_resolution.mp4 file and this gets stored as Encode object (chunk=False). This is what is available for download.

Apparently the Encode object is used to store Encoded files that are served eventually (chunk=False, status='success'), but also files while they are on their way to get transcoded (chunk=True, status='pending/etc')

(Parenthesis opening) there is also an experimental small service (not commited to the repo currently) that speaks only through API and a) gets tasks to run, b) returns results. So it makes a request and receives an ffmpeg command, plus a file, it runs the ffmpeg command, and returns the result.I've used this mechanism on a number of installations to migrate existing videos through more servers/cpu and has worked with only one problem, some temporary files needed to be removed from the servers (through a periodic task, not so big problem) (Parenthesis closing)

When the Encode object is marked as success and chunk=False, and thus is available for download/stream, there is a task that gets started and saves an HLS version of the file (1 mp4-->x number of small .ts chunks). This would be FILES_C

This mechanism allows for workers that have access on the same filesystem (either localhost, or through a shared network filesystem, eg NFS/EFS) to work on the same time and produce results.