6. Generation

This section refers to the core wiki page generation.

Every weewiki page generated corresponds to a top-level header.

For simplification purposes, a program in this context is considered to be a collection of top-level headers. If there are no level 1 headers, no pages are created. Any text that occurs before the first header will be skipped.

Page generation is a matter of getting the top-level id, and then iterating through all the individual components until it reaches the end. Along the way, it is determined where major sections start and end, and pages are broken up and generated accordingly.

<<page_generation>>=

<<find_first_id>>
<<find_last_id>>
<<iterate_through_components>>

The first header in the program needs to be found. This is the header with the smallest reference ID.

This can be found with wmp_header_top.

single_page mode is triggered happens when there are no headers to be found (and therefore no headers). Instead of breaking the page into sections, it will render everything into one page.

<<find_first_id>>=

id = wmp_header_top(core, NULL, prog);
if (id == 0) {
    single_page = 1;
    id = wmp_resource_top(core, NULL, prog);
}
page_id = id;

The last significant id of the program is found. When this resource id is reached, the program willl break. This is found with the function wmp_resource_last.

<<find_last_id>>=

last = wmp_resource_last(core, NULL, prog);

Org code gets rendered by iterating through headers, content, and block references.

<<iterate_through_components>>=

while (1) {
    wmp_resource res;
<<get_type>>
<<handle_component>>
<<check_for_last>>
<<update_id>>
}
<<flush_last_page>>

Document components are connected together as a linked list. Every one of these components has a "next" value, containing the reference ID of the next object.

Iteration through the list will continue to happen until the last ID is found. There, it will break the loop.

<<check_for_last>>=

if (id >= last) break;

The next value is (presumably) found from one of the components, it is updated at the end of the loop.

<<update_id>>=

if (next == 0) {
    /* debug code, this message shouldn't happen! */
    printf("next id is 0, coming from %d\n", id);
}
id = next;

The type of the reference ID is determined, and then the information is extracted from the right table.

<<get_type>>=

wmp_resource_init(&res);
rc = wmp_find_resource(core, id, &res, prog);

if (!rc) {
    fprintf(stderr, "Could not find resource %d\n", id);
    err = 1;
    goto cleanup;
}

Content gets appended to the working string in different ways depending on the type. There are three major types to consider: content data, headers, and code blocks.

For now: headers, content, and block references are hard coded as integers 3, 4, 5 (as seen in the enum defined in db.org). A less brittle solution may someday be implemented (enums + macros, perhaps?).

<<handle_component>>=

switch (res.type) {
    case 3:
<<append_header>>
        break;
    case 4:
<<append_content>>
        break;
    case 5:
<<append_block>>
        break;
    default:
        fprintf(stderr, "Not sure how to handle type %d\n",
                res.type);
        err = 1;
        goto cleanup;
}

Components get appended onto the end of a string as org code.

For content, it's a matter of appending the text as-is. This is the most straightforward.

<<append_content>>=

{
    wmp_content ct;
    rc = wmp_content_find(core, id, &ct, prog);

    if (!rc) {
        fprintf(stderr,
                "Could not find content %d in program %d\n",
                id,
                prog);
        err = 1;
        goto cleanup;
    }

    wwstring_append(&str, ct.content, strlen(ct.content));
    next = ct.next;
    wmp_content_free(&ct);
}

Headers require some processing. First, the header level is applied (the number of stars). Following that, the dynamically generated section number. Finally, the actual name itself is appended.

Following each header is a marker command from Janet. This is used to enable jump links for specific sections. The id used will be the relative worgmap id.

<<append_header>>=

{
    wmp_header hd;
    char tmp[16]; /* hope 16 levels is enough heh */
    int i;
    int level;
    char idstr[12]; /* wm_XXX_YYYY */

    rc = wmp_header_find(core, id, &hd, prog);

    if (!rc) {
        fprintf(stderr,
                "Could not find content %d in program %d\n",
                id,
                prog);
    }

<<check_for_new_section>>

    if (hd.level >= 14) level = 14;
    else level = hd.level;

    for (i = 0; i < hd.level; i++) {
        tmp[i] = '*';
    }

    tmp[level] = ' ';
    tmp[level + 1] = '\0';

    wwstring_append(&str, tmp, level + 1);
    wwstring_append(&str, hd.section, strlen(hd.section));
    wwstring_append(&str, " ", 1);
    wwstring_append(&str, hd.name, strlen(hd.name));
    wwstring_append(&str, "\n", 1);

    sprintf(idstr, "wm_%03d_%04d", prog, hd.id);
    idstr[11] = 0;
    wwstring_append(&str, "@!(marker \"", 11);
    wwstring_append(&str, idstr, 11);
    wwstring_append(&str, "\")!@\n", 5);

    next = hd.next;
    prev_header_id = hd.id;
    wmp_header_free(&hd);
}

Markers are referenced using inline janet function called marker. This creates an id reference that jump links can use.

A header marker generated with the format wm_PROG_ID where PROG is the program number, and ID is the reference id.

Generating code blocks is where things start to get interesting. A code block first pops up as a block reference, and is used to make a marker with the name wm_PROG_ID. From the block reference, the code block itself can be extracted. The subblock can then be recreated using the pos, ref, prev_lastseg, and segoffvalues.

A code subblock is a chain of segments. that gets written inside of a subblock. Segments are either piece of text, or block references. Block references will eventually turn into hyperlinks that go to a block page. For now, they will be represented in text form.

The first thing supplied here is a block reference. From the block reference, the actual named code block can be retrieved. This is found using wmp_blkref_codeblock, which returns the subblock as a list of segments.

<<append_block>>=

{
    wmp_blkref br;
    wmp_segment *segs;
    int nsegs;
    wmp_block blk;
    int k;

    nsegs = 0;
    wmp_blkref_init(&br);

    wmp_blkref_find(core, id, &br, prog);
    wmp_block_init(&blk);
    wmp_find_block(core, br.ref, &blk, prog);
    wmp_blkref_codeblock(core, &br, &segs, &nsegs);

    wwstring_append(&str, "#+NAME: ", 8);
    wwstring_append(&str, blk.name, strlen(blk.name));
    wwstring_append(&str, "\n", 1);
    wwstring_append(&str, "#+BEGIN_SRC c", 13);
    wwstring_append(&str, "\n", 1);
    for (k = 0; k < nsegs; k++) {
        if (segs[k].type == 0) {
            wwstring_append(&str, segs[k].str, strlen(segs[k].str));
        } else if (segs[k].type == 1) {
            wwstring_append(&str, "<<", 2);
            wwstring_append(&str, segs[k].str, strlen(segs[k].str));
            wwstring_append(&str, ">>", 2);
            wwstring_append(&str, "\n", 1);
        }
    }
    wwstring_append(&str, "#+END_SRC", 9);
    wwstring_append(&str, "\n", 1);

    next = br.next;
    wmp_blkref_free(&br);
    wmp_block_free(&blk);
    wmp_blkref_codeblock_free(core, &segs, nsegs);
}

A check is done to see if a page needs to be written. A new page can be written when a new major section is found (this will probably be set with some sort of flag).

<<check_for_new_section>>=

if (hd.level == 1 && prev_header_id > 0) {
<<create_new_wikipage>>
    wwstring_free(&str);
    wwstring_init(&str);
    page_id = hd.id;
}

<<create_new_wikipage>>=

rc = create_new_wikipage(wiki, prog, page_id, &str);
if (rc) {
    err = 1;
    goto cleanup;
}

A function called create_new_wikipage will create a new weewiki page given the database, program, page id, and content stored in a wwstring. This is needed as a function because it is called in more than one place.

<<helper_functions>>=

static int create_new_wikipage(sqlite3 *wiki,
                               int prog,
                               int page_id,
                               wwstring *str)
{
    char pgname[16]; /* wm_XXX_XXXX */
    int err;
    err = 0;
<<generate_page_name>>
<<append_footer>>
<<sql_insert_operation>>
    return err;
}

Creating a new weewiki is a matter of inserting a new row into the wiki table. A unique page name is created with the format WM_PROG_ID where PROG is the program ID, and ID is the resource ID associated with the top-level header.

<<generate_page_name>>=

sprintf(pgname, "wm_%03d_%04d", prog, page_id);

The data for page content itself is stored in a string that has been appended to since the last page was created.

The key/value pair for an operation is written via a SQL INSERT operation via the SQLite API.

<<sql_insert_operation>>=

{
    sqlite3_stmt *stmt;
    int rc;

    sqlite3_prepare_v2(wiki,
                       "INSERT INTO wiki"
                       "(key, value)\n"
                       "VALUES(?1, ?2);",
                       -1,
                       &stmt,
                       NULL);
    sqlite3_bind_text(stmt, 1, pgname, -1, NULL);
    sqlite3_bind_text(stmt, 2, str->str, -1, NULL);
    rc = sqlite3_step(stmt);
    if (rc != SQLITE_DONE) {
        sqlite3_finalize(stmt);
        fprintf(stderr, "Error: %s\n", sqlite3_errmsg(wiki));
        err = 1;
    }
    sqlite3_finalize(stmt);
}

Before rendering, a dynamically footer is appened to the end of the page before it is inserted into the weewiki database. To maximize flexibility, this is done as a call to a user-defined inline janet function wm-footer. The idea here is to provide prev/home/next page navigation. To do this, the top-level page id is needed, as well as the program id.

<<append_footer>>=

{
    char b[16];
    int sz;

    wwstring_append(str, "\n\n", 2);
    wwstring_append(str, "@!(wm-footer ", 13);
    sz = sprintf(b, "%d ", prog);
    wwstring_append(str, b, sz);
    sz = sprintf(b, "%d", page_id);
    wwstring_append(str, b, sz);
    wwstring_append(str, ")!@", 3);
}

At the end of the parsing, the last page must be written to disk, if there is a last page. Prior to this, the only way a new wiki page would be written was when a new major section occured. No more major sections following means this page would otherwise be stick in limbo.

<<flush_last_page>>=

if (single_page || (prev_header_id > 0 && str.sz > 0)) {
    rc = create_new_wikipage(wiki, prog, page_id, &str);

    if (rc) {
        err = 1;
        goto cleanup;
    }
}

prev | home | next