Reading content from an "unrelated" Github repository

Support Plugins
1

My goal is to publish multiple books/manuals inside of an existing Grav site.

  • Each book and manual is managed as an independent Github repository and its structure is defined by a project file (json/yaml)
  • The books will be multilingual.

Here are the details:

I have create a Grav site:

http://impagina.org/

  • The core gets updated through bin/gpm
  • The theme is pulled in through webhooks from https://github.com/impagina/htdocs-grav-theme
  • The pages are pulled in from https://github.com/impagina/htdocs-grav-pages

(the two links above are not clickable, since I’m not allowed to put more than two links in my posts…)

The first book that should published is: https://github.com/aoloe/scribus-book-starting-with.

The basic idea is that a webhook will trigger a pull and update the markdown files that the Grav plugin can then “intelligently” serve as a book.

Has anybody already done something similar? Any hints how I can achieve this?

Here are some more details:

  • I’m an experienced PHP programmer.
  • The structure of the book can be modified to better suit Grav.
  • I don’t wish to use the directory names for the order of the chapters (01.abcd, 02.efgh, …; reordering is too “hard”)
  • Putting all chapter’s file in separate directories is possible.
  • The content will be translated in multiple languages. (The Grav site will also be translated, but the book might be in more languages than the website).
  • Chapter might be missing or be outdated in some languages.
  • The .md files should display correctly (mind the images!) in the web view for the Github repository (but it will be “optimized” for Grav).
  • Multiple “chapters” might be rendered in one Grav page.
  • The resulting Grav plugin (if any must be created) will be open source.

I don’t mind collaborating with other people and create a solution that can be published in the plugin directory.

2

I am working - time permitting the distribution of focus between projects - on something similar, at least publishing books is a subset of it. Achieving this is “merely” a matter of coupling together enough solutions to handle every aspect of getting and putting the data into the structure desired. A few questions:

  • Does project.yaml define the structure in its entirety, exhaustively covering content and settings?
  • Using ordered directory-names does reduce the amount of logic needed for ordering, it would be easier to accommodate a better sorting-mechanism in Admin or the like. Is the structure of the books set to a defined standard?
  • Handling the translations should be easy enough, both book and site can follow the current language and fall back through defaults. Are you also interested in versioning combined with multi-language?
    • This is something I am working on currently, but it might require some more advanced taxonomical logic.
  • What, if anything, is currently determined about the content’s settings? That is, how does the renderer know which chapters may be output on the same page?

I have solutions for most of this, but for the project I am working on – also open source by the way – I have yet to decide on a unified standard for how to structure and inter-relate long content. Have you evaluated any standards like this?

The web is abundant with them, the problem is really what level of specificity they require. Markdown for content is fine as is, but the lowest possible specificity by the standard – without falling into ambiguity with bloated addons – is desirable to maintain portability and extensibility.

3

A few replies : - )

  • the current project.yaml is just a place holder. i think that i will need multiple “project” definitions for the different targets. examples could be project-grav.yaml and project-epub.yaml.
  • i don’t plan to manage/edit the content through grav. the content comes from a “normal” git repository and is only displayed in grav (the main idea is to pople people to contribute without having to give them any write rights and being able to check the quality of the contributions (pull requests) before they are published). and i want them to contribute to the book / manual, not to the website.
  • if i understand you correctly, versioning is something that could be interesting for my use case. but at this point i tend to solve it by forcing every content change to go through its own ticket.
  • it should not be hard to define a project file that allows aggregating multiple pages in one rendered page… i think.

i have not evaluated any standard, i don’t plan to create one (i would “simply” define a structure for my project), but if there is any standard that can be used for a book structure, i would be glad to use it! (if possible json or yaml based).

I have setup a repository where I store my thoughts and where I will eventually publish the resulting plugin (probably, I will have to change the name of the plugin at some time):

is there any place where i could read about what you already did?

4

A few thoughts on replies:

  • A decent standard for each project would be handy, something that determines how each part of the book fits together. I’ve looked at a few, but definitely YAML is preferable for its simplicity.
  • Even if managed entirely through Git, an organization that allows for easy editing is also desirable. It’s one of the reasons I mainly develop with Grav; I can transpose most content into and out of it.
  • Both multi-language and versioning would be directed by Git, just rendered through Grav.
  • Aggregation can be accomplished with a setting or two in FrontMatter (often combined with YAML), or even in the project-file.

My main objective in finding a decent standard, that is compliant with the flat-file, easy to navigate and flexible structure of Grav’s Pages, is that a well-defined standard is easier to translate into other standards. Thus Grav can be used as the renderer, but the project isn’t locked into it.

My current “todo” for the project is here: https://github.com/OleVik/grav-theme-scholar#todo. It’s currently a re-implementation of the Learn-theme, but I also scoped it for regular pages, blogging, papers, articles, and books. Largely, the idea is to liberate the process of writing and publishing academic content from locked-in systems.

5

Voilà, I’ve uploaded a proof of concept!

The book (work in progress…) stored in this repository

“defined” by this project file

is rendered on http://impagina.org as

http://impagina.org/learn/starting-with-scribus

and

http://impagina.org/learn/introduction-a-scribus

For obvious reaons, there are no public links to the two pages yet.

I got there by defining two Grav pages:

and

which are using the plugin

It’s not production ready but it works!

Comments and suggestions for improvement are very welcome!
I will be travelling to the mountains later today so, depending on the weather and on the access to internet, i will not be able to reply until next monday.

6

After the “Urrah” post above, here a few things, I’m not really happy about:

  • I could not find a good way to do the routing in the plugin. Currently, I’m defining two routes in site.yaml. (https://github.com/impagina/htdocs-grav-config/blob/master/site.yaml). It does work, but I’d really prefer to

    • hav the php code automatically manage the routing,
    • or at least being able to add the routes to the plugin/page configs and not to the site.

    Any hint?

  • I could not get Grav to tell me the relative path to the current page.

          $relativeMediaPath = substr($page->media()->path(), strlen(dirname($_SERVER['SCRIPT_FILENAME'])));

    Does not look like a nice solution to me (but it works).

    Is there a built-in solution?

  • I will have to figure out how to:

    • add the rendered files to the cache.
    • clear the cache (for the affected pages) when the book is updated on Github.

    Do you know of any example I can get inspiration from?

7

On top of it, I got two times a strange session error when jumping directly to the Book.

Going back to the /learn page and again to the book makes the problem disappear…

Ok, probably understood the issue: If I load the two pages in parallel in the same browser, then I get the error… It seems to be a locking issue… (you will get it if you are too fast when opening the book in new tabs)