Zettelstore

Copyright (c) 2020-present Detlef Stern
Copyright (c) 2020-2023 Detlef Stern

                          Licensed under the EUPL

Zettelstore is licensed under the European Union Public License, version 1.2 or
later (EUPL v. 1.2). The license is available in the official languages of the
EU. The English version is included here. Please see
https://joinup.ec.europa.eu/community/eupl/og_page/eupl for official

## Copyright (c) 2020-present Detlef Stern
## Copyright (c) 2020-2022 Detlef Stern
##
## This file is part of Zettelstore.
##
## Zettelstore is licensed under the latest version of the EUPL (European Union
## Public License). Please see file LICENSE.txt for your rights and obligations
## under this license.

often connects to Zettelstore via its API. Some of the software packages may be
experimental.

The software, including the manual, is licensed
under the [European Union Public License 1.2 (or
later)](https://zettelstore.de/home/file?name=LICENSE.txt&ci=trunk).

[Stay tuned](https://twitter.com/search?q=%40t73fde%20zettelstore&f=live) …
[Stay](https://twitter.com/zettelstore)
<a rel="me" href="https://nerdculture.de/@zettelstore">tuned</a>&nbsp;&hellip;

0.12-dev
0.10.1

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package ast provides the abstract syntax tree for parsed zettel content.
package ast

import (
	"net/url"

	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
)

// ZettelNode is the root node of the abstract syntax tree.
// It is *not* part of the visitor pattern.
type ZettelNode struct {
	Meta    *meta.Meta     // Original metadata
	Content zettel.Content // Original content
	Content domain.Content // Original content
	Zid     id.Zid         // Zettel identification.
	InhMeta *meta.Meta     // Metadata of the zettel, with inherited values.
	Ast     BlockSlice     // Zettel abstract syntax tree is a sequence of block nodes.
	Syntax  string         // Syntax / parser that produced the Ast
}

// Node is the interface, all nodes must implement.

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2023 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package ast

import (
	"net/url"
	"strings"

	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/domain/id"
)

// QueryPrefix is the prefix that denotes a query expression.
const QueryPrefix = "query:"

// ParseReference parses a string and returns a reference.
func ParseReference(s string) *Reference {

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package auth provides services for authentification / authorization.
package auth

import (
	"time"

	"zettelstore.de/z/box"
	"zettelstore.de/z/config"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
)

// BaseManager allows to check some base auth modes.
type BaseManager interface {
	// IsReadonly returns true, if the systems is configured to run in read-only-mode.
	IsReadonly() bool
}

// TokenKind specifies for which application / usage a token is/was requested.
type TokenKind int

// Allowed values of token kind
const (
	_ TokenKind = iota
	KindAPI
	KindwebUI
	KindJSON
	KindHTML
)

// TokenData contains some important elements from a token.
type TokenData struct {
	Token   []byte
	Now     time.Time
	Issued  time.Time

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package cred provides some function for handling credentials.
package cred

import (
	"bytes"

	"golang.org/x/crypto/bcrypt"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/domain/id"
)

// HashCredential returns a hashed vesion of the given credential
func HashCredential(zid id.Zid, ident, credential string) (string, error) {
	fullCredential := createFullCredential(zid, ident, credential)
	res, err := bcrypt.GenerateFromPassword(fullCredential, bcrypt.DefaultCost)
	if err != nil {

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

	"github.com/pascaldekloe/jwt"

	"zettelstore.de/c/api"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/auth/policy"
	"zettelstore.de/z/box"
	"zettelstore.de/z/config"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/kernel"
)

type myAuth struct {
	readonly bool
	owner    id.Zid
	secret   []byte
}

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package policy

import (
	"zettelstore.de/z/auth"
	"zettelstore.de/z/config"
	"zettelstore.de/z/zettel/meta"
	"zettelstore.de/z/domain/meta"
)

type anonPolicy struct {
	authConfig config.AuthConfig
	pre        auth.Policy
}

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package policy

import (
	"context"

	"zettelstore.de/z/auth"
	"zettelstore.de/z/box"
	"zettelstore.de/z/config"
	"zettelstore.de/z/query"
	"zettelstore.de/z/web/server"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/query"
	"zettelstore.de/z/web/server"
)

// BoxWithPolicy wraps the given box inside a policy box.
func BoxWithPolicy(
	manager auth.AuthzManager,
	box box.Box,
	authConfig config.AuthConfig,

	return pp.box.Location()
}

func (pp *polBox) CanCreateZettel(ctx context.Context) bool {
	return pp.box.CanCreateZettel(ctx)
}

func (pp *polBox) CreateZettel(ctx context.Context, zettel zettel.Zettel) (id.Zid, error) {
func (pp *polBox) CreateZettel(ctx context.Context, zettel domain.Zettel) (id.Zid, error) {
	user := server.GetUser(ctx)
	if pp.policy.CanCreate(user, zettel.Meta) {
		return pp.box.CreateZettel(ctx, zettel)
	}
	return id.Invalid, box.NewErrNotAllowed("Create", user, id.Invalid)
}

func (pp *polBox) GetZettel(ctx context.Context, zid id.Zid) (zettel.Zettel, error) {
	z, err := pp.box.GetZettel(ctx, zid)
func (pp *polBox) GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error) {
	zettel, err := pp.box.GetZettel(ctx, zid)
	if err != nil {
		return zettel.Zettel{}, err
		return domain.Zettel{}, err
	}
	user := server.GetUser(ctx)
	if pp.policy.CanRead(user, z.Meta) {
		return z, nil
	if pp.policy.CanRead(user, zettel.Meta) {
		return zettel, nil
	}
	return zettel.Zettel{}, box.NewErrNotAllowed("GetZettel", user, zid)
	return domain.Zettel{}, box.NewErrNotAllowed("GetZettel", user, zid)
}

func (pp *polBox) GetAllZettel(ctx context.Context, zid id.Zid) ([]zettel.Zettel, error) {
func (pp *polBox) GetAllZettel(ctx context.Context, zid id.Zid) ([]domain.Zettel, error) {
	return pp.box.GetAllZettel(ctx, zid)
}

func (pp *polBox) GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) {
	m, err := pp.box.GetMeta(ctx, zid)
	if err != nil {
		return nil, err

func (pp *polBox) SelectMeta(ctx context.Context, q *query.Query) ([]*meta.Meta, error) {
	user := server.GetUser(ctx)
	canRead := pp.policy.CanRead
	q = q.SetPreMatch(func(m *meta.Meta) bool { return canRead(user, m) })
	return pp.box.SelectMeta(ctx, q)
}

func (pp *polBox) CanUpdateZettel(ctx context.Context, zettel zettel.Zettel) bool {
func (pp *polBox) CanUpdateZettel(ctx context.Context, zettel domain.Zettel) bool {
	return pp.box.CanUpdateZettel(ctx, zettel)
}

func (pp *polBox) UpdateZettel(ctx context.Context, zettel zettel.Zettel) error {
func (pp *polBox) UpdateZettel(ctx context.Context, zettel domain.Zettel) error {
	zid := zettel.Meta.Zid
	user := server.GetUser(ctx)
	if !zid.IsValid() {
		return &box.ErrInvalidID{Zid: zid}
	}
	// Write existing zettel
	oldMeta, err := pp.box.GetMeta(ctx, zid)

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package policy

import (
	"zettelstore.de/c/api"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/zettel/meta"
	"zettelstore.de/z/domain/meta"
)

type defaultPolicy struct {
	manager auth.AuthzManager
}

func (*defaultPolicy) CanCreate(_, _ *meta.Meta) bool { return true }

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package policy

import (
	"zettelstore.de/c/api"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/config"
	"zettelstore.de/z/zettel/meta"
	"zettelstore.de/z/domain/meta"
)

type ownerPolicy struct {
	manager    auth.AuthzManager
	authConfig config.AuthConfig
	pre        auth.Policy
}

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package policy provides some interfaces and implementation for authorizsation policies.
package policy

import (
	"zettelstore.de/z/auth"
	"zettelstore.de/z/config"
	"zettelstore.de/z/zettel/meta"
	"zettelstore.de/z/domain/meta"
)

// newPolicy creates a policy based on given constraints.
func newPolicy(manager auth.AuthzManager, authConfig config.AuthConfig) auth.Policy {
	var pol auth.Policy
	if manager.IsReadonly() {
		pol = &roPolicy{}

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package policy

import (
	"fmt"
	"testing"

	"zettelstore.de/c/api"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
)

func TestPolicies(t *testing.T) {
	t.Parallel()
	testScene := []struct {
		readonly bool
		withAuth bool

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package policy

import "zettelstore.de/z/zettel/meta"
import "zettelstore.de/z/domain/meta"

type roPolicy struct{}

func (*roPolicy) CanCreate(_, _ *meta.Meta) bool   { return false }
func (*roPolicy) CanRead(_, _ *meta.Meta) bool     { return true }
func (*roPolicy) CanWrite(_, _, _ *meta.Meta) bool { return false }
func (*roPolicy) CanRename(_, _ *meta.Meta) bool   { return false }
func (*roPolicy) CanDelete(_, _ *meta.Meta) bool   { return false }
func (*roPolicy) CanRefresh(user *meta.Meta) bool  { return user != nil }

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package box provides a generic interface to zettel boxes.
package box

import (
	"context"
	"errors"
	"fmt"
	"io"
	"net/url"
	"strconv"
	"time"

	"zettelstore.de/c/api"
	"zettelstore.de/z/query"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/query"
)

// BaseBox is implemented by all Zettel boxes.
type BaseBox interface {
	// Location returns some information where the box is located.
	// Format is dependent of the box.
	Location() string

	// CanCreateZettel returns true, if box could possibly create a new zettel.
	CanCreateZettel(ctx context.Context) bool

	// CreateZettel creates a new zettel.
	// Returns the new zettel id (and an error indication).
	CreateZettel(ctx context.Context, zettel zettel.Zettel) (id.Zid, error)
	CreateZettel(ctx context.Context, zettel domain.Zettel) (id.Zid, error)

	// GetZettel retrieves a specific zettel.
	GetZettel(ctx context.Context, zid id.Zid) (zettel.Zettel, error)
	GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error)

	// GetMeta retrieves just the meta data of a specific zettel.
	GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error)

	// CanUpdateZettel returns true, if box could possibly update the given zettel.
	CanUpdateZettel(ctx context.Context, zettel zettel.Zettel) bool
	CanUpdateZettel(ctx context.Context, zettel domain.Zettel) bool

	// UpdateZettel updates an existing zettel.
	UpdateZettel(ctx context.Context, zettel zettel.Zettel) error
	UpdateZettel(ctx context.Context, zettel domain.Zettel) error

	// AllowRenameZettel returns true, if box will not disallow renaming the zettel.
	AllowRenameZettel(ctx context.Context, zid id.Zid) bool

	// RenameZettel changes the current Zid to a new Zid.
	RenameZettel(ctx context.Context, curZid, newZid id.Zid) error

	// ReadOnly indicates that the content of a box cannot change.
	ReadOnly bool

	// Zettel is the number of zettel managed by the box.
	Zettel int
}

// StartState enumerates the possible states of starting and stopping a box.
//
// StartStateStopped -> StartStateStarting -> StartStateStarted -> StateStateStopping -> StartStateStopped.
//
// Other transitions are also possible.
type StartState uint8

// Constant values of StartState
const (
	StartStateStopped StartState = iota
	StartStateStarting
	StartStateStarted
	StartStateStopping
)

// StartStopper performs simple lifecycle management.
type StartStopper interface {
	// State the current status of the box.
	State() StartState

	// Start the box. Now all other functions of the box are allowed.
	// Starting a box, which is not in state StartStateStopped is not allowed.
	// Starting an already started box is not allowed.
	Start(ctx context.Context) error

	// Stop the started box. Now only the Start() function is allowed.
	Stop(ctx context.Context)
}

// Refresher allow to refresh their internal data.

	// FetchZids returns the set of all zettel identifer managed by the box.
	FetchZids(ctx context.Context) (id.Set, error)

	// SelectMeta returns a list of metadata that comply to the given selection criteria.
	SelectMeta(ctx context.Context, q *query.Query) ([]*meta.Meta, error)

	// GetAllZettel retrieves a specific zettel from all managed boxes.
	GetAllZettel(ctx context.Context, zid id.Zid) ([]zettel.Zettel, error)
	GetAllZettel(ctx context.Context, zid id.Zid) ([]domain.Zettel, error)

	// GetAllMeta retrieves the meta data of a specific zettel from all managed boxes.
	GetAllMeta(ctx context.Context, zid id.Zid) ([]*meta.Meta, error)

	// Refresh the data from the box and from its managed sub-boxes.
	Refresh(context.Context) error
}

// UpdateReason gives an indication, why the ObserverFunc was called.
type UpdateReason uint8

// Values for Reason
const (
	_        UpdateReason = iota
	OnReady               // Box is started and fully operational
	OnReload              // Box was reloaded
	OnZettel              // Something with a zettel happened
)

// UpdateInfo contains all the data about a changed zettel.
type UpdateInfo struct {
	Box    BaseBox
	Box    Box
	Reason UpdateReason
	Zid    id.Zid
}

// UpdateFunc is a function to be called when a change is detected.
type UpdateFunc func(UpdateInfo)

// ErrCapacity is returned if a box has reached its capacity.
var ErrCapacity = errors.New("capacity exceeded")

// ErrInvalidID is returned if the zettel id is not appropriate for the box operation.
type ErrInvalidID struct{ Zid id.Zid }

func (err *ErrInvalidID) Error() string { return "invalid Zettel id: " + err.Zid.String() }

// GetQueryBool is a helper function to extract bool values from a box URI.
func GetQueryBool(u *url.URL, key string) bool {
	_, ok := u.Query()[key]
	return ok
}

// GetQueryInt is a helper function to extract int values of a specified range from a box URI.
func GetQueryInt(u *url.URL, key string, min, def, max int) int {
	sVal := u.Query().Get(key)
	if sVal == "" {
		return def
	}
	iVal, err := strconv.Atoi(sVal)
	if err != nil {
		return def
	}
	if iVal < min {
		return min
	}
	if iVal > max {
		return max
	}
	return iVal
}

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package compbox provides zettel that have computed content.
package compbox

import (
	"context"
	"net/url"

	"zettelstore.de/c/api"
	"zettelstore.de/z/box"
	"zettelstore.de/z/box/manager"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/logger"
	"zettelstore.de/z/query"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func init() {
	manager.Register(
		" comp",
		func(u *url.URL, cdata *manager.ConnectData) (box.ManagedBox, error) {
			return getCompBox(cdata.Number, cdata.Enricher), nil

// Setup remembers important values.
func Setup(cfg *meta.Meta) { myConfig = cfg.Clone() }

func (*compBox) Location() string { return "" }

func (*compBox) CanCreateZettel(context.Context) bool { return false }

func (cb *compBox) CreateZettel(context.Context, zettel.Zettel) (id.Zid, error) {
func (cb *compBox) CreateZettel(context.Context, domain.Zettel) (id.Zid, error) {
	cb.log.Trace().Err(box.ErrReadOnly).Msg("CreateZettel")
	return id.Invalid, box.ErrReadOnly
}

func (cb *compBox) GetZettel(_ context.Context, zid id.Zid) (zettel.Zettel, error) {
func (cb *compBox) GetZettel(_ context.Context, zid id.Zid) (domain.Zettel, error) {
	if gen, ok := myZettel[zid]; ok && gen.meta != nil {
		if m := gen.meta(zid); m != nil {
			updateMeta(m)
			if genContent := gen.content; genContent != nil {
				cb.log.Trace().Msg("GetMeta/Content")
				return zettel.Zettel{
				return domain.Zettel{
					Meta:    m,
					Content: zettel.NewContent(genContent(m)),
					Content: domain.NewContent(genContent(m)),
				}, nil
			}
			cb.log.Trace().Msg("GetMeta/NoContent")
			return zettel.Zettel{Meta: m}, nil
			return domain.Zettel{Meta: m}, nil
		}
	}
	cb.log.Trace().Err(box.ErrNotFound).Msg("GetZettel/Err")
	return zettel.Zettel{}, box.ErrNotFound
	return domain.Zettel{}, box.ErrNotFound
}

func (cb *compBox) GetMeta(_ context.Context, zid id.Zid) (*meta.Meta, error) {
	if gen, ok := myZettel[zid]; ok {
		if genMeta := gen.meta; genMeta != nil {
			if m := genMeta(zid); m != nil {
				updateMeta(m)

				handle(m)
			}
		}
	}
	return nil
}

func (*compBox) CanUpdateZettel(context.Context, zettel.Zettel) bool { return false }
func (*compBox) CanUpdateZettel(context.Context, domain.Zettel) bool { return false }

func (cb *compBox) UpdateZettel(context.Context, zettel.Zettel) error {
func (cb *compBox) UpdateZettel(context.Context, domain.Zettel) error {
	cb.log.Trace().Err(box.ErrReadOnly).Msg("UpdateZettel")
	return box.ErrReadOnly
}

func (*compBox) AllowRenameZettel(_ context.Context, zid id.Zid) bool {
	_, ok := myZettel[zid]
	return !ok

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package compbox

import (
	"bytes"

	"zettelstore.de/c/api"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/kernel"
)

func genConfigZettelM(zid id.Zid) *meta.Meta {
	if myConfig == nil {
		return nil
	}
	m := meta.New(zid)

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package compbox

import (
	"bytes"
	"fmt"

	"zettelstore.de/c/api"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/kernel"
)

func genKeysM(zid id.Zid) *meta.Meta {
	m := meta.New(zid)
	m.Set(api.KeyTitle, "Zettelstore Supported Metadata Keys")
	m.Set(api.KeyCreated, kernel.Main.GetConfig(kernel.CoreService, kernel.CoreVTime).(string))
	m.Set(api.KeyVisibility, api.ValueVisibilityLogin)

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package compbox

import (
	"bytes"

	"zettelstore.de/c/api"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/kernel"
)

func genLogM(zid id.Zid) *meta.Meta {
	m := meta.New(zid)
	m.Set(api.KeyTitle, "Zettelstore Log")
	m.Set(api.KeySyntax, meta.SyntaxText)
	m.Set(api.KeyCreated, kernel.Main.GetConfig(kernel.CoreService, kernel.CoreStarted).(string))

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package compbox

import (
	"bytes"
	"fmt"

	"zettelstore.de/c/api"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/kernel"
)

func genManagerM(zid id.Zid) *meta.Meta {
	m := meta.New(zid)
	m.Set(api.KeyTitle, "Zettelstore Box Manager")
	m.Set(api.KeyCreated, kernel.Main.GetConfig(kernel.CoreService, kernel.CoreStarted).(string))
	return m

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package compbox

import (
	"bytes"
	"fmt"
	"sort"
	"strings"

	"zettelstore.de/c/api"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/parser"
)

func genParserM(zid id.Zid) *meta.Meta {
	m := meta.New(zid)
	m.Set(api.KeyTitle, "Zettelstore Supported Parser")
	m.Set(api.KeyCreated, kernel.Main.GetConfig(kernel.CoreService, kernel.CoreVTime).(string))
	m.Set(api.KeyVisibility, api.ValueVisibilityLogin)

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package compbox

import (
	"zettelstore.de/c/api"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/kernel"
)

func getVersionMeta(zid id.Zid, title string) *meta.Meta {
	m := meta.New(zid)
	m.Set(api.KeyTitle, title)
	m.Set(api.KeyVisibility, api.ValueVisibilityExpert)
	return m

  input.zs-secondary { float:left }
  input.zs-upload {
    padding-left: 1em;
    padding-right: 1em;
  }
  a:not([class]) { text-decoration-skip-ink: auto }
  a.broken { text-decoration: line-through }
  a.external::after { content: "➚"; display: inline-block }
  img { max-width: 100% }
  img.right { float: right }
  ol.zs-endnotes {
    padding-top: .5rem;
    border-top: 1px solid;
  }
  kbd { font-family:monospace }

<link rel="stylesheet" href="{{{CSSUserURL}}}">
{{#CSSRoleURL}}<link rel="stylesheet" href="{{{CSSRoleURL}}}">{{/CSSRoleURL}}
<title>{{Title}}</title>
</head>
<body>
<nav class="zs-menu">
<a href="{{{HomeURL}}}">Home</a>
{{#WithAuth}}
{{#WithUser}}
<div class="zs-dropdown">
<button>User</button>
<nav class="zs-dropdown-content">
{{#WithAuth}}
{{#UserIsValid}}
<a href="{{{UserZettelURL}}}">{{UserIdent}}</a>
<a href="{{{LogoutURL}}}">Logout</a>
{{/UserIsValid}}
{{^UserIsValid}}
<a href="{{{LoginURL}}}">Login</a>
{{/UserIsValid}}
{{#UserIsValid}}
<a href="{{{LogoutURL}}}">Logout</a>
{{/UserIsValid}}
{{/WithAuth}}
</nav>
</div>
{{/WithAuth}}
{{/WithUser}}
<div class="zs-dropdown">
<button>Lists</button>
<nav class="zs-dropdown-content">
<a href="{{{ListZettelURL}}}">List Zettel</a>
<a href="{{{ListRolesURL}}}">List Roles</a>
<a href="{{{ListTagsURL}}}">List Tags</a>
{{#CanRefresh}}

(@@@@
(html ,@(if lang `((@ (lang ,lang))))
(head
  (meta (@ (charset "utf-8")))
  (meta (@ (name "viewport") (content "width=device-width, initial-scale=1.0")))
  (meta (@ (name "generator") (content "Zettelstore")))
  (meta (@ (name "format-detection") (content "telephone=no")))
  ,@META-HEADER
  (link (@ (rel "stylesheet") (href ,css-base-url)))
  (link (@ (rel "stylesheet") (href ,css-user-url)))
  ,@(if css-role-url `((link (@ (rel "stylesheet") (href ,css-role-url)))))
  (title ,title))
(body
  (nav (@ (class "zs-menu"))
    (a (@ (href ,home-url)) "Home")
    ,@(if with-auth
      `((div (@ (class "zs-dropdown"))
        (button "User")
        (nav (@ (class "zs-dropdown-content"))
          ,@(if user-is-valid
            `((a (@ (href ,user-zettel-url)) ,user-ident)
              (a (@ (href ,logout-url)) "Logout"))
            `((a (@ (href ,login-url)) "Login"))
          )
      )))
    )
    (div (@ (class "zs-dropdown"))
      (button "Lists")
      (nav (@ (class "zs-dropdown-content"))
        (a (@ (href ,list-zettel-url)) "List Zettel")
        (a (@ (href ,list-roles-url)) "List Roles")
        (a (@ (href ,list-tags-url)) "List Tags")
        ,@(if can-refresh `((a (@ (href ,refresh-url)) "Refresh")))
    ))
    ,@(if new-zettel-links
      `((div (@ (class "zs-dropdown"))
        (button "New")
        (nav (@ (class "zs-dropdown-content"))
          ,@(map pair-to-href new-zettel-links)
       )))
    )
    (form (@ (action ,search-url))
      (input (@ (type "text") (placeholder "Search..") (name ,query-key-query))))
  )
  (main (@ (class "content")) ,DETAIL)
  ,@(if FOOTER `((footer (hr) ,@FOOTER)))
  ,@(if debug-mode '((div (b "WARNING: Debug mode is enabled. DO NOT USE IN PRODUCTION!"))))
)))

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package constbox puts zettel inside the executable.
package constbox

import (
	"context"
	_ "embed" // Allow to embed file content
	"net/url"

	"zettelstore.de/c/api"
	"zettelstore.de/z/box"
	"zettelstore.de/z/box/manager"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/logger"
	"zettelstore.de/z/query"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func init() {
	manager.Register(
		" const",
		func(u *url.URL, cdata *manager.ConnectData) (box.ManagedBox, error) {
			return &constBox{
				log: kernel.Main.GetLogger(kernel.BoxService).Clone().
					Str("box", "const").Int("boxnum", int64(cdata.Number)).Child(),
				number:   cdata.Number,
				zettel:   constZettelMap,
				enricher: cdata.Enricher,
			}, nil
		})
}

type constHeader map[string]string

type constZettel struct {
	header  constHeader
	content zettel.Content
	content domain.Content
}

type constBox struct {
	log      *logger.Logger
	number   int
	zettel   map[id.Zid]constZettel
	enricher box.Enricher
}

func (*constBox) Location() string { return "const:" }

func (*constBox) CanCreateZettel(context.Context) bool { return false }

func (cb *constBox) CreateZettel(context.Context, zettel.Zettel) (id.Zid, error) {
func (cb *constBox) CreateZettel(context.Context, domain.Zettel) (id.Zid, error) {
	cb.log.Trace().Err(box.ErrReadOnly).Msg("CreateZettel")
	return id.Invalid, box.ErrReadOnly
}

func (cb *constBox) GetZettel(_ context.Context, zid id.Zid) (zettel.Zettel, error) {
func (cb *constBox) GetZettel(_ context.Context, zid id.Zid) (domain.Zettel, error) {
	if z, ok := cb.zettel[zid]; ok {
		cb.log.Trace().Msg("GetZettel")
		return zettel.Zettel{Meta: meta.NewWithData(zid, z.header), Content: z.content}, nil
		return domain.Zettel{Meta: meta.NewWithData(zid, z.header), Content: z.content}, nil
	}
	cb.log.Trace().Err(box.ErrNotFound).Msg("GetZettel")
	return zettel.Zettel{}, box.ErrNotFound
	return domain.Zettel{}, box.ErrNotFound
}

func (cb *constBox) GetMeta(_ context.Context, zid id.Zid) (*meta.Meta, error) {
	if z, ok := cb.zettel[zid]; ok {
		cb.log.Trace().Msg("GetMeta")
		return meta.NewWithData(zid, z.header), nil
	}

			cb.enricher.Enrich(ctx, m, cb.number)
			handle(m)
		}
	}
	return nil
}

func (*constBox) CanUpdateZettel(context.Context, zettel.Zettel) bool { return false }
func (*constBox) CanUpdateZettel(context.Context, domain.Zettel) bool { return false }

func (cb *constBox) UpdateZettel(context.Context, zettel.Zettel) error {
func (cb *constBox) UpdateZettel(context.Context, domain.Zettel) error {
	cb.log.Trace().Err(box.ErrReadOnly).Msg("UpdateZettel")
	return box.ErrReadOnly
}

func (cb *constBox) AllowRenameZettel(_ context.Context, zid id.Zid) bool {
	_, ok := cb.zettel[zid]
	return !ok

		constHeader{
			api.KeyTitle:      "Zettelstore Runtime Configuration",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxNone,
			api.KeyCreated:    "20200804111624",
			api.KeyVisibility: api.ValueVisibilityOwner,
		},
		zettel.NewContent(nil)},
		domain.NewContent(nil)},
	id.MustParse(api.ZidLicense): {
		constHeader{
			api.KeyTitle:      "Zettelstore License",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxText,
			api.KeyCreated:    "20210504135842",
			api.KeyLang:       api.ValueLangEN,
			api.KeyModified:   "20220131153422",
			api.KeyReadOnly:   api.ValueTrue,
			api.KeyVisibility: api.ValueVisibilityPublic,
		},
		zettel.NewContent(contentLicense)},
		domain.NewContent(contentLicense)},
	id.MustParse(api.ZidAuthors): {
		constHeader{
			api.KeyTitle:      "Zettelstore Contributors",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxZmk,
			api.KeyCreated:    "20210504135842",
			api.KeyLang:       api.ValueLangEN,
			api.KeyReadOnly:   api.ValueTrue,
			api.KeyVisibility: api.ValueVisibilityLogin,
		},
		zettel.NewContent(contentContributors)},
		domain.NewContent(contentContributors)},
	id.MustParse(api.ZidDependencies): {
		constHeader{
			api.KeyTitle:      "Zettelstore Dependencies",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxZmk,
			api.KeyLang:       api.ValueLangEN,
			api.KeyReadOnly:   api.ValueTrue,
			api.KeyVisibility: api.ValueVisibilityLogin,
			api.KeyCreated:    "20210504135842",
			api.KeyModified:   "20221013105100",
		},
		zettel.NewContent(contentDependencies)},
		domain.NewContent(contentDependencies)},
	id.BaseTemplateZid + 30000: {
		constHeader{
			api.KeyTitle:      "Zettelstore Base HTML Template (Mustache)",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxMustache,
			api.KeyCreated:    "20210504135842",
			api.KeyModified:   "20230523142249",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentBaseMustache)},
	id.BaseTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Base HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20230510155100",
			api.KeySyntax:     meta.SyntaxMustache,
			api.KeyCreated:    "20210504135842",
			api.KeyModified:   "20230523144403",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentBaseSxn)},
		domain.NewContent(contentBaseMustache)},
	id.LoginTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Login Form HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeySyntax:     meta.SyntaxMustache,
			api.KeyCreated:    "20200804111624",
			api.KeyModified:   "20230527144100",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentLoginSxn)},
		domain.NewContent(contentLoginMustache)},
	id.ZettelTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Zettel HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20230510155300",
			api.KeySyntax:     meta.SyntaxMustache,
			api.KeyCreated:    "20200804111624",
			api.KeyModified:   "20230523212800",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentZettelSxn)},
		domain.NewContent(contentZettelMustache)},
	id.InfoTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Info HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxMustache,
			api.KeyCreated:    "20200804111624",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentInfoMustache)},
		domain.NewContent(contentInfoMustache)},
	id.ContextTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Context HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxMustache,
			api.KeyCreated:    "20210218181140",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		domain.NewContent(contentContextMustache)},
	id.FormTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Form HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeySyntax:     meta.SyntaxMustache,
			api.KeyCreated:    "20200804111624",
			api.KeyModified:   "20230528210200",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentFormSxn)},
		domain.NewContent(contentFormMustache)},
	id.RenameTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Rename Form HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeySyntax:     meta.SyntaxMustache,
			api.KeyCreated:    "20200804111624",
			api.KeyModified:   "20230528174800",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentRenameSxn)},
		domain.NewContent(contentRenameMustache)},
	id.DeleteTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Delete HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeySyntax:     meta.SyntaxMustache,
			api.KeyCreated:    "20200804111624",
			api.KeyModified:   "20230528154900",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentDeleteSxn)},
		domain.NewContent(contentDeleteMustache)},
	id.ListTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore List Zettel HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20230526221600",
			api.KeySyntax:     meta.SyntaxMustache,
			api.KeyCreated:    "20200804111624",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentListZettelSxn)},
		domain.NewContent(contentListZettelMustache)},
	id.ErrorTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Error HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeySyntax:     meta.SyntaxMustache,
			api.KeyCreated:    "20210305133215",
			api.KeyModified:   "20230527224800",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentErrorSxn)},
		domain.NewContent(contentErrorMustache)},
	id.MustParse(api.ZidBaseCSS): {
		constHeader{
			api.KeyTitle:      "Zettelstore Base CSS",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxCSS,
			api.KeyCreated:    "20200804111624",
			api.KeyVisibility: api.ValueVisibilityPublic,
		},
		zettel.NewContent(contentBaseCSS)},
		domain.NewContent(contentBaseCSS)},
	id.MustParse(api.ZidUserCSS): {
		constHeader{
			api.KeyTitle:      "Zettelstore User CSS",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxCSS,
			api.KeyCreated:    "20210622110143",
			api.KeyVisibility: api.ValueVisibilityPublic,
		},
		zettel.NewContent([]byte("/* User-defined CSS */"))},
		domain.NewContent([]byte("/* User-defined CSS */"))},
	id.RoleCSSMapZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Role to CSS Map",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxNone,
			api.KeyCreated:    "20220321183214",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(nil)},
		domain.NewContent(nil)},
	id.EmojiZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Generic Emoji",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxGif,
			api.KeyReadOnly:   api.ValueTrue,
			api.KeyCreated:    "20210504175807",
			api.KeyVisibility: api.ValueVisibilityPublic,
		},
		zettel.NewContent(contentEmoji)},
		domain.NewContent(contentEmoji)},
	id.TOCNewTemplateZid: {
		constHeader{
			api.KeyTitle:      "New Menu",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxZmk,
			api.KeyLang:       api.ValueLangEN,
			api.KeyCreated:    "20210217161829",
			api.KeyVisibility: api.ValueVisibilityCreator,
		},
		zettel.NewContent(contentNewTOCZettel)},
		domain.NewContent(contentNewTOCZettel)},
	id.MustParse(api.ZidTemplateNewZettel): {
		constHeader{
			api.KeyTitle:      "New Zettel",
			api.KeyRole:       api.ValueRoleZettel,
			api.KeySyntax:     meta.SyntaxZmk,
			api.KeyCreated:    "20201028185209",
			api.KeyVisibility: api.ValueVisibilityCreator,
		},
		zettel.NewContent(nil)},
		domain.NewContent(nil)},
	id.MustParse(api.ZidTemplateNewUser): {
		constHeader{
			api.KeyTitle:                       "New User",
			api.KeyRole:                        api.ValueRoleConfiguration,
			api.KeySyntax:                      meta.SyntaxNone,
			api.KeyCreated:                     "20201028185209",
			meta.NewPrefix + api.KeyCredential: "",
			meta.NewPrefix + api.KeyUserID:     "",
			meta.NewPrefix + api.KeyUserRole:   api.ValueUserRoleReader,
			api.KeyVisibility:                  api.ValueVisibilityOwner,
		},
		zettel.NewContent(nil)},
		domain.NewContent(nil)},
	id.DefaultHomeZid: {
		constHeader{
			api.KeyTitle:   "Home",
			api.KeyRole:    api.ValueRoleZettel,
			api.KeySyntax:  meta.SyntaxZmk,
			api.KeyLang:    api.ValueLangEN,
			api.KeyCreated: "20210210190757",
		},
		zettel.NewContent(contentHomeZettel)},
		domain.NewContent(contentHomeZettel)},
}

//go:embed license.txt
var contentLicense []byte

//go:embed contributors.zettel
var contentContributors []byte

//go:embed dependencies.zettel
var contentDependencies []byte

//go:embed base.mustache
var contentBaseMustache []byte

//go:embed base.sxn
var contentBaseSxn []byte

//go:embed login.sxn
var contentLoginSxn []byte
//go:embed login.mustache
var contentLoginMustache []byte

//go:embed zettel.sxn
var contentZettelSxn []byte
//go:embed zettel.mustache
var contentZettelMustache []byte

//go:embed info.mustache
var contentInfoMustache []byte

//go:embed context.mustache
var contentContextMustache []byte

//go:embed form.sxn
var contentFormSxn []byte
//go:embed form.mustache
var contentFormMustache []byte

//go:embed rename.sxn
var contentRenameSxn []byte
//go:embed rename.mustache
var contentRenameMustache []byte

//go:embed delete.sxn
var contentDeleteSxn []byte
//go:embed delete.mustache
var contentDeleteMustache []byte

//go:embed listzettel.sxn
var contentListZettelSxn []byte
//go:embed listzettel.mustache
var contentListZettelMustache []byte

//go:embed error.sxn
var contentErrorSxn []byte
//go:embed error.mustache
var contentErrorMustache []byte

//go:embed base.css
var contentBaseCSS []byte

//go:embed emoji_spin.gif
var contentEmoji []byte

//go:embed newtoc.zettel
var contentNewTOCZettel []byte

//go:embed home.zettel
var contentHomeZettel []byte

<header>
<h1>{{Title}}</h1>
<div class="zs-meta">
<a href="{{{InfoURL}}}">Info</a>
&#183; <a href="?dir=backward">Backward</a>
&#183; <a href="?dir=both">Both</a>
&#183; <a href="?dir=forward">Forward</a>
&#183; Cost:{{#Costs}}&#x2000;<a href="{{{URL}}}">{{{Text}}}</a>{{/Costs}}
</div>
</header>
{{{Content}}}

<article>
<header>
<h1>Delete Zettel {{Zid}}</h1>
</header>
<p>Do you really want to delete this zettel?</p>
{{#HasShadows}}
<div class="zs-info">
<h2>Infomation</h2>
<p>If you delete this zettel, the previoulsy shadowed zettel from overlayed box {{ShadowedBox}} becomes available.</p>
</div>
{{/HasShadows}}
{{#Incoming.Has}}
<div class="zs-warning">
<h2>Warning!</h2>
<p>If you delete this zettel, incoming references from the following zettel will become invalid.</p>
<ul>
{{#Incoming.Links}}
<li><a href="{{{URL}}}">{{Text}}</a></li>
{{/Incoming.Links}}
</ul>
</div>
{{/Incoming.Has}}
{{#HasUselessFiles}}
<div class="zs-warning">
<h2>Warning!</h2>
<p>Deleting this zettel will also delete the following files, so that they will not be interpreted as content for this zettel.</p>
<ul>
{{#UselessFiles}}
<li>{{{.}}}</li>
{{/UselessFiles}}
</ul>
</div>
{{/HasUselessFiles}}
<dl>
{{#MetaPairs}}
<dt>{{Key}}:</dt><dd>{{Value}}</dd>
{{/MetaPairs}}
</dl>
<form method="POST">
<input class="zs-primary" type="submit" value="Delete">
</form>
</article>
{{end}}

(article
  (header (h1 "Delete Zettel " ,zid))
  (p "Do you really want to delete this zettel?")
  ,@(if shadowed-box
    `((div (@ (class "zs-info"))
      (h2 "Information")
      (p "If you delete this zettel, the previously shadowed zettel from overlayed box " ,shadowed-box " becomes available.")
    ))
  )
  ,@(if incoming
    `((div (@ (class "zs-warning"))
      (h2 "Warning!")
      (p "If you delete this zettel, incoming references from the following zettel will become invalid.")
      (ul ,@(map pair-to-href-li incoming))
    ))
  )
  ,@(if useless
    `((div (@ (class "zs-warning"))
      (h2 "Warning!")
      (p "Deleting this zettel will also delete the following files, so that they will not be interpreted as content for this zettel.")
      (ul ,@(map (lambda (s) `(li ,s)) useless))
    ))
  )
  ,(pairs-to-dl meta-pairs)
  (form (@ (method "POST")) (input (@ (class "zs-primary") (type "submit") (value "Delete"))))
)

<article>
<header>
<h1>{{ErrorTitle}}</h1>
</header>
{{ErrorText}}
</article>

(article
  (header (h1 ,heading))
  ,message
)

<article>
<header>
<h1>{{Heading}}</h1>
</header>
<form action="{{FormActionURL}}" method="POST" enctype="multipart/form-data">
<div>
<label for="zs-title">Title <a title="Main heading of this zettel. You can use inline zettelmarkup.">&#9432;</a></label>
<input class="zs-input" type="text" id="zs-title" name="title" placeholder="Title.." value="{{MetaTitle}}" autofocus>
</div>
<div>
<div>
<label for="zs-role">Role <a title="One word, without spaces, to set the main role of this zettel.">&#9432;</a></label>
<input class="zs-input" type="text" id="zs-role" {{#HasRoleData}}list="zs-role-data"{{/HasRoleData}} name="role" placeholder="role.." value="{{MetaRole}}">
{{#HasRoleData}}
<datalist id="zs-role-data">
{{#RoleData}}
<option value="{{.}}">
{{/RoleData}}
</datalist>
{{/HasRoleData}}
</div>
<label for="zs-tags">Tags <a title="Tags must begin with an '#' sign. They are separated by spaces.">&#9432;</a></label>
<input class="zs-input" type="text" id="zs-tags" name="tags" placeholder="#tag" value="{{MetaTags}}">
</div>
<div>
<label for="zs-meta">Metadata <a title="Other metadata for this zettel. Each line contains a key/value pair, separated by a colon ':'.">&#9432;</a></label>
<textarea class="zs-input" id="zs-meta" name="meta" rows="4" placeholder="metakey: metavalue">
{{#MetaPairsRest}}
{{Key}}: {{Value}}
{{/MetaPairsRest}}
</textarea>
</div>
<div>
<label for="zs-syntax">Syntax <a title="Syntax of zettel content below, one word. Typically 'zmk' (for zettelmarkup).">&#9432;</a></label>
<input class="zs-input" type="text" id="zs-syntax" {{#HasSyntaxData}}list="zs-syntax-data"{{/HasSyntaxData}} name="syntax" placeholder="syntax.." value="{{MetaSyntax}}">
{{#HasSyntaxData}}
<datalist id="zs-syntax-data">
{{#SyntaxData}}
<option value="{{.}}">
{{/SyntaxData}}
</datalist>
{{/HasSyntaxData}}</div>
<div>
{{#IsTextContent}}
<label for="zs-content">Content <a title="Content for this zettel, according to above syntax.">&#9432;</a></label>
<textarea class="zs-input zs-content" id="zs-content" name="content" rows="20" placeholder="Your content..">{{Content}}</textarea>
{{/IsTextContent}}
</div>
<div>
<input class="zs-primary" type="submit" value="Submit">
<input class="zs-secondary" type="submit" value="Save" formaction="?save">
<input class="zs-upload" type="file" id="zs-file" name="file">
</div>
</form>
</article>

(article
  (header (h1 ,heading))
  (form (@ (action ,form-action-url) (method "POST") (enctype "multipart/form-data"))
  (div
    (label (@ (for "zs-title")) "Title " (a (@ (title "Main heading of this zettel.")) (@H "ⓘ")))
    (input (@ (class "zs-input") (type "text") (id "zs-title") (name "title") (placeholder "Title..") (value ,meta-title) (autofocus))))
  (div
    (label (@ (for "zs-role")) "Role " (a (@ (title "One word, without spaces, to set the main role of this zettel.")) (@H "ⓘ")))
    (input (@ (class "zs-input") (type "text") (id "zs-role") (name "role") (placeholder "role..") (value ,meta-role)
      ,@(if role-data '((list "zs-role-data")))
    ))
    ,@(if role-data
      `((datalist (@ (id "zs-role-data"))
        ,@(map (lambda (v) `(option (@ (value ,v)))) role-data)
      ))
    )
  )
  (div
    (label (@ (for "zs-tags")) "Tags " (a (@ (title "Tags must begin with an '#' sign. They are separated by spaces.")) (@H "ⓘ")))
    (input (@ (class "zs-input") (type "text") (id "zs-tags") (name "tags") (placeholder "#tag") (value ,meta-tags))))
  (div
    (label (@ (for "zs-meta")) "Metadata " (a (@ (title "Other metadata for this zettel. Each line contains a key/value pair, separated by a colon ':'.")) (@H "ⓘ")))
    (textarea (@ (class "zs-input") (id "zs-meta") (name "meta") (rows "4") (placeholder "metakey: metavalue")) ,meta))
  (div
    (label (@ (for "zs-syntax")) "Syntax " (a (@ (title "Syntax of zettel content below, one word. Typically 'zmk' (for zettelmarkup).")) (@H "ⓘ")))
    (input (@ (class "zs-input") (type "text") (id "zs-syntax") (name "syntax") (placeholder "syntax..") (value ,meta-syntax)
      ,@(if syntax-data '((list "zs-syntax-data")))
    ))
    ,@(if syntax-data
      `((datalist (@ (id "zs-syntax-data"))
        ,@(map (lambda (v) `(option (@ (value ,v)))) syntax-data)
      ))
    )
  )
  ,@(if (bound? 'content)
    `((div
      (label (@ (for "zs-content")) "Content " (a (@ (title "Content for this zettel, according to above syntax.")) (@H "ⓘ")))
      (textarea (@ (class "zs-input zs-content") (id "zs-content") (name "content") (rows "20") (placeholder "Zettel content..")) ,content)
    ))
  )
  (div
    (input (@ (class "zs-primary") (type "submit") (value "Submit")))
    (input (@ (class "zs-secondary") (type "submit") (value "Save") (formaction "?save")))
    (input (@ (class "zs-upload") (type "file") (id "zs-file") (name "file")))
  ))
)

</tr>
{{/ParseMatrix}}
</table>
{{#HasShadowLinks}}
<h2>Shadowed Boxes</h2>
<ul>{{#ShadowLinks}}<li>{{.}}</li>{{/ShadowLinks}}</ul>
{{/HasShadowLinks}}
{{#Endnotes}}{{{Endnotes}}}{{/Endnotes}}
</article>

Copyright (c) 2020-present Detlef Stern
Copyright (c) 2020-2023 Detlef Stern

                          Licensed under the EUPL

Zettelstore is licensed under the European Union Public License, version 1.2 or
later (EUPL v. 1.2). The license is available in the official languages of the
EU. The English version is included here. Please see
https://joinup.ec.europa.eu/community/eupl/og_page/eupl for official

<header>
<h1>{{Title}}</h1>
</header>
<form action="{{{SearchURL}}}">
<input class="zs-input" type="text" placeholder="Search.." name="{{QueryKeyQuery}}" value="{{QueryValue}}">
</form>
{{{Content}}}
{{#CanCreate}}<form action="{{{CreateURL}}}">
<input type="hidden" name="{{QueryKeyQuery}}" value="{{QueryValue}}">
<input type="hidden" name="{{QueryKeySeed}}" value="{{Seed}}">
<input class="zs-primary" type="submit" value="Save As Zettel">
</form>{{/CanCreate}}

(article
  (header (h1 ,heading))
  (form (@ (action ,search-url))
    (input (@ (class "zs-input") (type "text") (placeholder "Search..") (name ,query-key-query) (value ,query-value))))
  ,@content
  ,@endnotes
  ,@(if can-create
    `((form (@ (action ,create-url))
      (input (@ (type "hidden") (name ,query-key-query) (value ,query-value)))
      (input (@ (type "hidden") (name ,query-key-seed) (value ,seed)))
      (input (@ (class "zs-primary") (type "submit") (value "Save As Zettel")))
    ))
  )
)

<article>
<header>
<h1>{{Title}}</h1>
</header>
{{#Retry}}
<div class="zs-indication zs-error">Wrong user name / password. Try again.</div>
{{/Retry}}
<form method="POST" action="">
<div>
<label for="username">User name:</label>
<input class="zs-input" type="text" id="username" name="username" placeholder="Your user name.." autofocus>
</div>
<div>
<label for="password">Password:</label>
<input class="zs-input" type="password" id="password" name="password" placeholder="Your password..">
</div>
<div><input class="zs-primary" type="submit" value="Login"></div>
</form>
</article>

(article
  (header (h1 ,heading))
  ,@(if retry '((div (@ (class "zs-indication zs-error")) "Wrong user name / password. Try again.")))
  (form (@ (method "POST") (action ""))
    (div
      (label (@ (for "username")) "User name:")
      (input (@ (class "zs-input") (type "text") (id "username") (name "username") (placeholder "Your user name..") (autofocus))))
    (div
      (label (@ (for "password")) "Password:")
      (input (@ (class "zs-input") (type "password") (id "password") (name "password") (placeholder "Your password.."))))
    (div
      (input (@ (class "zs-primary") (type "submit") (value "Login"))))
  )
)

<article>
<header>
<h1>Rename Zettel {{Zid}}</h1>
</header>
<p>Do you really want to rename this zettel?</p>
{{#Incoming.Has}}
<div class="zs-warning">
<h2>Warning!</h2>
<p>If you rename this zettel, incoming references from the following zettel will become invalid.</p>
<ul>
{{#Incoming.Links}}
<li><a href="{{{URL}}}">{{Text}}</a></li>
{{/Incoming.Links}}
</ul>
</div>
{{/Incoming.Has}}
{{#HasUselessFiles}}
<div class="zs-warning">
<h2>Warning!</h2>
<p>Renaming this zettel will also delete the following files, so that they will not be interpreted as content for a zettel with identifier {{Zid}}.</p>
<ul>
{{#UselessFiles}}
<li>{{{.}}}</li>
{{/UselessFiles}}
</ul>
</div>
{{/HasUselessFiles}}
<form method="POST">
<div>
<label for="newid">New zettel id</label>
<input class="zs-input" type="text" id="newzid" name="newzid" placeholder="ZID.." value="{{Zid}}" autofocus>
</div>
<input type="hidden" id="curzid" name="curzid" value="{{Zid}}">
<div><input class="zs-primary" type="submit" value="Rename"></div>
</form>
<dl>
{{#MetaPairs}}
<dt>{{Key}}:</dt><dd>{{Value}}</dd>
{{/MetaPairs}}
</dl>
</article>

(article
  (header (h1 "Rename Zettel " ,zid))
  (p "Do you really want to rename this zettel?")
  ,@(if incoming
    `((div (@ (class "zs-warning"))
      (h2 "Warning!")
      (p "If you rename this zettel, incoming references from the following zettel will become invalid.")
      (ul ,@(map pair-to-href-li incoming))
    ))
  )
  ,@(if useless
    `((div (@ (class "zs-warning"))
      (h2 "Warning!")
      (p "Renaming this zettel will also delete the following files, so that they will not be interpreted as content for this zettel.")
      (ul ,@(map (lambda (s) `(li ,s)) useless))
    ))
  )
  (form (@ (method "POST"))
    (input (@ (type "hidden" (id "curzid") (name "curzid") (value ,zid))))
    (div
      (label (@ (for "newid") "New zettel id"))
      (input (@ (class "zs-input") (type "text") (id "newid") (name "newid") (placeholder "ZID..") (value ,zid) (autofocus))))
    (div (input (@ (class "zs-primary") (type "submit") (value "Rename"))))
  )
  ,(pairs-to-dl meta-pairs)
)

<article>
<header>
<h1>{{{HTMLTitle}}}</h1>
<div class="zs-meta">
{{#CanWrite}}<a href="{{{EditURL}}}">Edit</a> &#183;{{/CanWrite}}
{{Zid}} &#183;
<a href="{{{InfoURL}}}">Info</a> &#183;
(<a href="{{{RoleURL}}}">{{RoleText}}</a>)
{{#Tags.Has}}&#183; {{#Tags.Links}} <a href="{{{URL}}}">{{Text}}</a>{{/Tags.Links}}{{/Tags.Has}}
{{#CanCopy}}&#183; <a href="{{{CopyURL}}}">Copy</a>{{/CanCopy}}
{{#CanVersion}}&#183; <a href="{{{VersionURL}}}">Version</a>{{/CanVersion}}
{{#CanFolge}}&#183; <a href="{{{FolgeURL}}}">Folge</a>{{/CanFolge}}
{{#PredecessorRefs}}<br>Predecessor: {{{PredecessorRefs}}}{{/PredecessorRefs}}
{{#PrecursorRefs}}<br>Precursor: {{{PrecursorRefs}}}{{/PrecursorRefs}}
{{#HasExtURL}}<br>URL: <a href="{{{ExtURL}}}"{{{ExtNewWindow}}}>{{ExtURL}}</a>{{/HasExtURL}}
{{#Author}}<br>By {{Author}}{{/Author}}
</div>
</header>
{{{Content}}}
</article>
{{#NeedBottomNav}}<nav>{{/NeedBottomNav}}
{{#FolgeLinks.Has}}
<details open>
<summary>Folgezettel</summary>
<ul>
{{#FolgeLinks.Links}}
<li><a href="{{{URL}}}">{{Text}}</a></li>
{{/FolgeLinks.Links}}
</ul>
</details>
{{/FolgeLinks.Has}}
{{#BackLinks.Has}}
<details open>
<summary>Incoming</summary>
<ul>
{{#BackLinks.Links}}
<li><a href="{{{URL}}}">{{Text}}</a></li>
{{/BackLinks.Links}}
</ul>
</details>
{{/BackLinks.Has}}
{{#SuccessorLinks.Has}}
<details open>
<summary>Successors</summary>
<ul>
{{#SuccessorLinks.Links}}
<li><a href="{{{URL}}}">{{Text}}</a></li>
{{/SuccessorLinks.Links}}
</ul>
</details>
{{/SuccessorLinks.Has}}
{{#NeedBottomNav}}</nav>{{/NeedBottomNav}}

(article
  (header
    (h1 ,heading)
    (div (@ (class "zs-meta"))
      ,@(if can-write `((a (@ (href ,edit-url)) "Edit") (@H " · ")))
      ,zid (@H " · ")
      (a (@ (href ,info-url)) "Info") (@H " · ")
      "(" (a (@ (href ,role-url)) ,meta-role) ")"
      ,@(if tag-refs `((@H " · ") ,@tag-refs))
      ,@(if can-copy `((@H " · ") (a (@ (href ,copy-url)) "Copy")))
      ,@(if can-version `((@H " · ") (a (@ (href ,version-url)) "Version")))
      ,@(if can-folge `((@H " · ") (a (@ (href ,folge-url)) "Folge")))
      ,@(if predecessor-refs `((br) "Predecessor: " ,predecessor-refs))
      ,@(if precursor-refs `((br) "Precursor: " ,precursor-refs))
      ,@(if superior-refs `((br) "Superior: " ,superior-refs))
      ,@(if ext-url `((br) "URL: " ,ext-url))
      ,@(let (author (and (bound? 'meta-author) meta-author)) (if author `((br) "By " ,author)))
    )
  )
  ,@content
  ,endnotes
  ,@(if (or folge-links subordinate-links back-links successor-links)
    `((nav
      ,@(if folge-links `((details (@ (open)) (summary "Folgezettel") (ul ,@(map pair-to-href-li folge-links)))))
      ,@(if subordinate-links `((details (@ (open)) (summary "Subordinates") (ul ,@(map pair-to-href-li subordinate-links)))))
      ,@(if back-links `((details (@ (open)) (summary "Incoming") (ul ,@(map pair-to-href-li back-links)))))
      ,@(if successor-links `((details (@ (open)) (summary "Successors") (ul ,@(map pair-to-href-li successor-links)))))
     ))
  )
)

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2023 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

	"os"
	"path/filepath"
	"sync"

	"zettelstore.de/z/box"
	"zettelstore.de/z/box/manager"
	"zettelstore.de/z/box/notify"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/logger"
	"zettelstore.de/z/query"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func init() {
	manager.Register("dir", func(u *url.URL, cdata *manager.ConnectData) (box.ManagedBox, error) {
		var log *logger.Logger
		if krnl := kernel.Main; krnl != nil {
			log = krnl.GetLogger(kernel.BoxService).Clone().Str("box", "dir").Int("boxnum", int64(cdata.Number)).Child()

	mxCmds     sync.RWMutex
}

func (dp *dirBox) Location() string {
	return dp.location
}

func (dp *dirBox) State() box.StartState {
	if ds := dp.dirSrv; ds != nil {
		switch ds.State() {
		case notify.DsCreated:
			return box.StartStateStopped
		case notify.DsStarting:
			return box.StartStateStarting
		case notify.DsWorking:
			return box.StartStateStarted
		case notify.DsMissing:
			return box.StartStateStarted
		case notify.DsStopping:
			return box.StartStateStopping
		}
	}
	return box.StartStateStopped
}

func (dp *dirBox) Start(context.Context) error {
	dp.mxCmds.Lock()
	defer dp.mxCmds.Unlock()
	dp.fCmds = make([]chan fileCmd, 0, dp.fSrvs)
	for i := uint32(0); i < dp.fSrvs; i++ {
		cc := make(chan fileCmd)
		go fileService(i, dp.log.Clone().Str("sub", "file").Uint("fn", uint64(i)).Child(), dp.dir, cc)

	}
	if err != nil {
		dp.log.Fatal().Err(err).Msg("Unable to create directory supervisor")
		dp.stopFileServices()
		return err
	}
	dp.dirSrv = notify.NewDirService(
		dp,
		dp.log.Clone().Str("sub", "dirsrv").Child(),
		notifier,
		dp.cdata.Notify,
	)
	dp.dirSrv.Start()
	return nil
}

	return dp.fCmds[sum%dp.fSrvs]
}

func (dp *dirBox) CanCreateZettel(_ context.Context) bool {
	return !dp.readonly
}

func (dp *dirBox) CreateZettel(ctx context.Context, zettel zettel.Zettel) (id.Zid, error) {
func (dp *dirBox) CreateZettel(ctx context.Context, zettel domain.Zettel) (id.Zid, error) {
	if dp.readonly {
		return id.Invalid, box.ErrReadOnly
	}

	newZid, err := dp.dirSrv.SetNewDirEntry()
	if err != nil {
		return id.Invalid, err

		err = dp.dirSrv.UpdateDirEntry(&entry)
	}
	dp.notifyChanged(meta.Zid)
	dp.log.Trace().Err(err).Zid(meta.Zid).Msg("CreateZettel")
	return meta.Zid, err
}

func (dp *dirBox) GetZettel(ctx context.Context, zid id.Zid) (zettel.Zettel, error) {
func (dp *dirBox) GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error) {
	entry := dp.dirSrv.GetDirEntry(zid)
	if !entry.IsValid() {
		return zettel.Zettel{}, box.ErrNotFound
		return domain.Zettel{}, box.ErrNotFound
	}
	m, c, err := dp.srvGetMetaContent(ctx, entry, zid)
	if err != nil {
		return zettel.Zettel{}, err
		return domain.Zettel{}, err
	}
	zettel := zettel.Zettel{Meta: m, Content: zettel.NewContent(c)}
	zettel := domain.Zettel{Meta: m, Content: domain.NewContent(c)}
	dp.log.Trace().Zid(zid).Msg("GetZettel")
	return zettel, nil
}

func (dp *dirBox) GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) {
	m, err := dp.doGetMeta(ctx, zid)
	dp.log.Trace().Zid(zid).Err(err).Msg("GetMeta")

		}
		dp.cdata.Enricher.Enrich(ctx, m, dp.number)
		handle(m)
	}
	return nil
}

func (dp *dirBox) CanUpdateZettel(context.Context, zettel.Zettel) bool {
func (dp *dirBox) CanUpdateZettel(context.Context, domain.Zettel) bool {
	return !dp.readonly
}

func (dp *dirBox) UpdateZettel(ctx context.Context, zettel zettel.Zettel) error {
func (dp *dirBox) UpdateZettel(ctx context.Context, zettel domain.Zettel) error {
	if dp.readonly {
		return box.ErrReadOnly
	}

	meta := zettel.Meta
	zid := meta.Zid
	if !zid.IsValid() {

	if err == nil {
		dp.notifyChanged(zid)
	}
	dp.log.Trace().Zid(zid).Err(err).Msg("UpdateZettel")
	return err
}

func (dp *dirBox) updateEntryFromMetaContent(entry *notify.DirEntry, m *meta.Meta, content zettel.Content) {
func (dp *dirBox) updateEntryFromMetaContent(entry *notify.DirEntry, m *meta.Meta, content domain.Content) {
	entry.SetupFromMetaContent(m, content, dp.cdata.Config.GetZettelFileSyntax)
}

func (dp *dirBox) AllowRenameZettel(context.Context, id.Zid) bool {
	return !dp.readonly
}

	}

	newEntry, err := dp.dirSrv.RenameDirEntry(curEntry, newZid)
	if err != nil {
		return err
	}
	oldMeta.Zid = newZid
	newZettel := zettel.Zettel{Meta: oldMeta, Content: zettel.NewContent(oldContent)}
	newZettel := domain.Zettel{Meta: oldMeta, Content: domain.NewContent(oldContent)}
	if err = dp.srvSetZettel(ctx, &newEntry, newZettel); err != nil {
		// "Rollback" rename. No error checking...
		dp.dirSrv.RenameDirEntry(&newEntry, curZid)
		return err
	}
	err = dp.srvDeleteZettel(ctx, curEntry, curZid)
	if err == nil {

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package dirbox

import (
	"context"
	"io"
	"os"
	"path/filepath"
	"time"

	"zettelstore.de/z/box/filebox"
	"zettelstore.de/z/box/notify"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/input"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/logger"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func fileService(i uint32, log *logger.Logger, dirPath string, cmds <-chan fileCmd) {
	// Something may panic. Ensure a running service.
	defer func() {
		if r := recover(); r != nil {
			kernel.Main.LogRecover("FileService", r)

	cmd.rc <- resGetMetaContent{m, content, err}
}

// COMMAND: srvSetZettel ----------------------------------------
//
// Writes a new or exsting zettel.

func (dp *dirBox) srvSetZettel(ctx context.Context, entry *notify.DirEntry, zettel zettel.Zettel) error {
func (dp *dirBox) srvSetZettel(ctx context.Context, entry *notify.DirEntry, zettel domain.Zettel) error {
	rc := make(chan resSetZettel, 1)
	dp.getFileChan(zettel.Meta.Zid) <- &fileSetZettel{entry, zettel, rc}
	ctx, cancel := context.WithTimeout(ctx, serviceTimeout)
	defer cancel()
	select {
	case err := <-rc:
		return err
	case <-ctx.Done():
		return ctx.Err()
	}
}

type fileSetZettel struct {
	entry  *notify.DirEntry
	zettel zettel.Zettel
	zettel domain.Zettel
	rc     chan<- resSetZettel
}
type resSetZettel = error

func (cmd *fileSetZettel) run(log *logger.Logger, dirPath string) {
	entry := cmd.entry
	zid := entry.Zid

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package filebox provides boxes that are stored in a file.
package filebox

import (
	"errors"
	"net/url"
	"path/filepath"
	"strings"

	"zettelstore.de/c/api"
	"zettelstore.de/z/box"
	"zettelstore.de/z/box/manager"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/kernel"
)

func init() {
	manager.Register("file", func(u *url.URL, cdata *manager.ConnectData) (box.ManagedBox, error) {
		path := getFilepathFromURL(u)
		ext := strings.ToLower(filepath.Ext(path))
		if ext != ".zip" {

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2023 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package filebox

import (
	"archive/zip"
	"context"
	"io"
	"strings"

	"zettelstore.de/z/box"
	"zettelstore.de/z/box/notify"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/input"
	"zettelstore.de/z/logger"
	"zettelstore.de/z/query"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

type zipBox struct {
	log      *logger.Logger
	number   int
	name     string
	enricher box.Enricher
	notify   chan<- box.UpdateInfo
	dirSrv   *notify.DirService
}

func (zb *zipBox) Location() string {
	if strings.HasPrefix(zb.name, "/") {
		return "file://" + zb.name
	}
	return "file:" + zb.name
}

func (zb *zipBox) State() box.StartState {
	if ds := zb.dirSrv; ds != nil {
		switch ds.State() {
		case notify.DsCreated:
			return box.StartStateStopped
		case notify.DsStarting:
			return box.StartStateStarting
		case notify.DsWorking:
			return box.StartStateStarted
		case notify.DsMissing:
			return box.StartStateStarted
		case notify.DsStopping:
			return box.StartStateStopping
		}
	}
	return box.StartStateStopped
}

func (zb *zipBox) Start(context.Context) error {
	reader, err := zip.OpenReader(zb.name)
	if err != nil {
		return err
	}
	reader.Close()
	zipNotifier := notify.NewSimpleZipNotifier(zb.log, zb.name)
	zb.dirSrv = notify.NewDirService(zb, zb.log, zipNotifier, zb.notify)
	zb.dirSrv = notify.NewDirService(zb.log, zipNotifier, zb.notify)
	zb.dirSrv.Start()
	return nil
}

func (zb *zipBox) Refresh(_ context.Context) {
	zb.dirSrv.Refresh()
	zb.log.Trace().Msg("Refresh")
}

func (zb *zipBox) Stop(context.Context) {
	zb.dirSrv.Stop()
	zb.dirSrv = nil
}

func (*zipBox) CanCreateZettel(context.Context) bool { return false }

func (zb *zipBox) CreateZettel(context.Context, zettel.Zettel) (id.Zid, error) {
func (zb *zipBox) CreateZettel(context.Context, domain.Zettel) (id.Zid, error) {
	err := box.ErrReadOnly
	zb.log.Trace().Err(err).Msg("CreateZettel")
	return id.Invalid, err
}

func (zb *zipBox) GetZettel(_ context.Context, zid id.Zid) (zettel.Zettel, error) {
func (zb *zipBox) GetZettel(_ context.Context, zid id.Zid) (domain.Zettel, error) {
	entry := zb.dirSrv.GetDirEntry(zid)
	if !entry.IsValid() {
		return zettel.Zettel{}, box.ErrNotFound
		return domain.Zettel{}, box.ErrNotFound
	}
	reader, err := zip.OpenReader(zb.name)
	if err != nil {
		return zettel.Zettel{}, err
		return domain.Zettel{}, err
	}
	defer reader.Close()

	var m *meta.Meta
	var src []byte
	var inMeta bool

	contentName := entry.ContentName
	if metaName := entry.MetaName; metaName == "" {
		if contentName == "" {
			zb.log.Panic().Zid(zid).Msg("No meta, no content in zipBox.GetZettel")
		}
		src, err = readZipFileContent(reader, entry.ContentName)
		if err != nil {
			return zettel.Zettel{}, err
			return domain.Zettel{}, err
		}
		if entry.HasMetaInContent() {
			inp := input.NewInput(src)
			m = meta.NewFromInput(zid, inp)
			src = src[inp.Pos:]
		} else {
			m = CalcDefaultMeta(zid, entry.ContentExt)
		}
	} else {
		m, err = readZipMetaFile(reader, zid, metaName)
		if err != nil {
			return zettel.Zettel{}, err
			return domain.Zettel{}, err
		}
		inMeta = true
		if contentName != "" {
			src, err = readZipFileContent(reader, entry.ContentName)
			if err != nil {
				return zettel.Zettel{}, err
				return domain.Zettel{}, err
			}
		}
	}

	CleanupMeta(m, zid, entry.ContentExt, inMeta, entry.UselessFiles)
	zb.log.Trace().Zid(zid).Msg("GetZettel")
	return zettel.Zettel{Meta: m, Content: zettel.NewContent(src)}, nil
	return domain.Zettel{Meta: m, Content: domain.NewContent(src)}, nil
}

func (zb *zipBox) GetMeta(_ context.Context, zid id.Zid) (*meta.Meta, error) {
	entry := zb.dirSrv.GetDirEntry(zid)
	if !entry.IsValid() {
		return nil, box.ErrNotFound
	}

		}
		zb.enricher.Enrich(ctx, m, zb.number)
		handle(m)
	}
	return nil
}

func (*zipBox) CanUpdateZettel(context.Context, zettel.Zettel) bool { return false }
func (*zipBox) CanUpdateZettel(context.Context, domain.Zettel) bool { return false }

func (zb *zipBox) UpdateZettel(context.Context, zettel.Zettel) error {
func (zb *zipBox) UpdateZettel(context.Context, domain.Zettel) error {
	err := box.ErrReadOnly
	zb.log.Trace().Err(err).Msg("UpdateZettel")
	return err
}

func (zb *zipBox) AllowRenameZettel(_ context.Context, zid id.Zid) bool {
	entry := zb.dirSrv.GetDirEntry(zid)

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package box

import (
	"net/url"
	"strconv"
	"time"

	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/domain/id"
)

// GetNewZid calculates a new and unused zettel identifier, based on the current date and time.
func GetNewZid(testZid func(id.Zid) (bool, error)) (id.Zid, error) {
	withSeconds := false
	for i := 0; i < 90; i++ { // Must be completed within 9 seconds (less than web/server.writeTimeout)
		zid := id.New(withSeconds)
		found, err := testZid(zid)
		if err != nil {
			return id.Invalid, err
		}
		if found {
			return zid, nil
		}
		// TODO: do not wait here unconditionally.
		time.Sleep(100 * time.Millisecond)
		withSeconds = true
	}
	return id.Invalid, ErrConflict
}

// GetQueryBool is a helper function to extract bool values from a box URI.
func GetQueryBool(u *url.URL, key string) bool {
	_, ok := u.Query()[key]
	return ok
}

// GetQueryInt is a helper function to extract int values of a specified range from a box URI.
func GetQueryInt(u *url.URL, key string, min, def, max int) int {
	sVal := u.Query().Get(key)
	if sVal == "" {
		return def
	}
	iVal, err := strconv.Atoi(sVal)
	if err != nil {
		return def
	}
	if iVal < min {
		return min
	}
	if iVal > max {
		return max
	}
	return iVal
}

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package manager

import (
	"sync"

	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/domain/id"
)

type arAction int

const (
	arNothing arAction = iota
	arReload

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package manager

import (
	"testing"

	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/domain/id"
)

func TestSimple(t *testing.T) {
	t.Parallel()
	ar := newAnterooms(2)
	ar.EnqueueZettel(id.Zid(1))
	action, zid, rno := ar.Dequeue()

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package manager

import (
	"bytes"
	"context"
	"errors"
	"strings"

	"zettelstore.de/z/box"
	"zettelstore.de/z/query"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/query"
)

// Conatains all box.Box related functions

// Location returns some information where the box is located.
func (mgr *Manager) Location() string {
	if len(mgr.boxes) <= 2 {
		return "NONE"
	}
	var sb strings.Builder
	var buf bytes.Buffer
	for i := 0; i < len(mgr.boxes)-2; i++ {
		if i > 0 {
			sb.WriteString(", ")
			buf.WriteString(", ")
		}
		sb.WriteString(mgr.boxes[i].Location())
		buf.WriteString(mgr.boxes[i].Location())
	}
	return sb.String()
	return buf.String()
}

// CanCreateZettel returns true, if box could possibly create a new zettel.
func (mgr *Manager) CanCreateZettel(ctx context.Context) bool {
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	return mgr.State() == box.StartStateStarted && mgr.boxes[0].CanCreateZettel(ctx)
	return mgr.started && mgr.boxes[0].CanCreateZettel(ctx)
}

// CreateZettel creates a new zettel.
func (mgr *Manager) CreateZettel(ctx context.Context, zettel zettel.Zettel) (id.Zid, error) {
func (mgr *Manager) CreateZettel(ctx context.Context, zettel domain.Zettel) (id.Zid, error) {
	mgr.mgrLog.Debug().Msg("CreateZettel")
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if mgr.State() != box.StartStateStarted {
	if !mgr.started {
		return id.Invalid, box.ErrStopped
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	return mgr.boxes[0].CreateZettel(ctx, zettel)
}

// GetZettel retrieves a specific zettel.
func (mgr *Manager) GetZettel(ctx context.Context, zid id.Zid) (zettel.Zettel, error) {
func (mgr *Manager) GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error) {
	mgr.mgrLog.Debug().Zid(zid).Msg("GetZettel")
	if mgr.State() != box.StartStateStarted {
		return zettel.Zettel{}, box.ErrStopped
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if !mgr.started {
		return domain.Zettel{}, box.ErrStopped
	}
	for i, p := range mgr.boxes {
		if z, err := p.GetZettel(ctx, zid); err != box.ErrNotFound {
			if err == nil {
				mgr.Enrich(ctx, z.Meta, i+1)
			}
			return z, err
		}
	}
	return zettel.Zettel{}, box.ErrNotFound
	return domain.Zettel{}, box.ErrNotFound
}

// GetAllZettel retrieves a specific zettel from all managed boxes.
func (mgr *Manager) GetAllZettel(ctx context.Context, zid id.Zid) ([]zettel.Zettel, error) {
func (mgr *Manager) GetAllZettel(ctx context.Context, zid id.Zid) ([]domain.Zettel, error) {
	mgr.mgrLog.Debug().Zid(zid).Msg("GetAllZettel")
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if mgr.State() != box.StartStateStarted {
	if !mgr.started {
		return nil, box.ErrStopped
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	var result []zettel.Zettel
	var result []domain.Zettel
	for i, p := range mgr.boxes {
		if z, err := p.GetZettel(ctx, zid); err == nil {
			mgr.Enrich(ctx, z.Meta, i+1)
			result = append(result, z)
		}
	}
	return result, nil
}

// GetMeta retrieves just the meta data of a specific zettel.
func (mgr *Manager) GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) {
	mgr.mgrLog.Debug().Zid(zid).Msg("GetMeta")
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if mgr.State() != box.StartStateStarted {
	if !mgr.started {
		return nil, box.ErrStopped
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	return mgr.doGetMeta(ctx, zid)
}

func (mgr *Manager) doGetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) {
	for i, p := range mgr.boxes {
		if m, err := p.GetMeta(ctx, zid); err != box.ErrNotFound {
			if err == nil {
				mgr.Enrich(ctx, m, i+1)
			}
			return m, err
		}
	}
	return nil, box.ErrNotFound
}

// GetAllMeta retrieves the meta data of a specific zettel from all managed boxes.
func (mgr *Manager) GetAllMeta(ctx context.Context, zid id.Zid) ([]*meta.Meta, error) {
	mgr.mgrLog.Debug().Zid(zid).Msg("GetAllMeta")
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if mgr.State() != box.StartStateStarted {
	if !mgr.started {
		return nil, box.ErrStopped
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	var result []*meta.Meta
	for i, p := range mgr.boxes {
		if m, err := p.GetMeta(ctx, zid); err == nil {
			mgr.Enrich(ctx, m, i+1)
			result = append(result, m)
		}
	}
	return result, nil
}

// FetchZids returns the set of all zettel identifer managed by the box.
func (mgr *Manager) FetchZids(ctx context.Context) (id.Set, error) {
	mgr.mgrLog.Debug().Msg("FetchZids")
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if mgr.State() != box.StartStateStarted {
	if !mgr.started {
		return nil, box.ErrStopped
	}
	result := id.Set{}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	for _, p := range mgr.boxes {
		err := p.ApplyZid(ctx, func(zid id.Zid) { result.Zid(zid) }, func(id.Zid) bool { return true })
		if err != nil {
			return nil, err
		}
	}
	return result, nil
}

type metaMap map[id.Zid]*meta.Meta

// SelectMeta returns all zettel meta data that match the selection
// criteria. The result is ordered by descending zettel id.
func (mgr *Manager) SelectMeta(ctx context.Context, q *query.Query) ([]*meta.Meta, error) {
	if msg := mgr.mgrLog.Debug(); msg.Enabled() {
		msg.Str("query", q.String()).Msg("SelectMeta")
	}
	if mgr.State() != box.StartStateStarted {
		return nil, box.ErrStopped
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	return mgr.doSelectMeta(ctx, q)
}
func (mgr *Manager) doSelectMeta(ctx context.Context, q *query.Query) ([]*meta.Meta, error) {
	compSearch, err := q.RetrieveAndCompile(ctx, mgr, mgr.doGetMeta, mgr.doSelectMeta)
	if err != nil {
		return nil, err
	if !mgr.started {
		return nil, box.ErrStopped
	}
	if result := compSearch.Result(); result != nil {
		mgr.mgrLog.Trace().Int("count", int64(len(result))).Msg("found without ApplyMeta")
		return result, nil
	}

	compSearch := q.RetrieveAndCompile(mgr)
	selected := metaMap{}
	for _, term := range compSearch.Terms {
		rejected := id.Set{}
		handleMeta := func(m *meta.Meta) {
			zid := m.Zid
			if rejected.Contains(zid) {
				mgr.mgrLog.Trace().Zid(zid).Msg("SelectMeta/alreadyRejected")

				mgr.mgrLog.Trace().Zid(zid).Msg("SelectMeta/match")
			} else {
				rejected.Zid(zid)
				mgr.mgrLog.Trace().Zid(zid).Msg("SelectMeta/reject")
			}
		}
		for _, p := range mgr.boxes {
			if err2 := p.ApplyMeta(ctx, handleMeta, term.Retrieve); err2 != nil {
				return nil, err2
			if err := p.ApplyMeta(ctx, handleMeta, term.Retrieve); err != nil {
				return nil, err
			}
		}
	}
	result := make([]*meta.Meta, 0, len(selected))
	for _, m := range selected {
		result = append(result, m)
	}
	result = compSearch.AfterSearch(result)
	mgr.mgrLog.Trace().Int("count", int64(len(result))).Msg("found with ApplyMeta")
	return result, nil
	return q.Sort(result), nil
}

// CanUpdateZettel returns true, if box could possibly update the given zettel.
func (mgr *Manager) CanUpdateZettel(ctx context.Context, zettel zettel.Zettel) bool {
func (mgr *Manager) CanUpdateZettel(ctx context.Context, zettel domain.Zettel) bool {
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	return mgr.State() == box.StartStateStarted && mgr.boxes[0].CanUpdateZettel(ctx, zettel)
	return mgr.started && mgr.boxes[0].CanUpdateZettel(ctx, zettel)
}

// UpdateZettel updates an existing zettel.
func (mgr *Manager) UpdateZettel(ctx context.Context, zettel zettel.Zettel) error {
func (mgr *Manager) UpdateZettel(ctx context.Context, zettel domain.Zettel) error {
	mgr.mgrLog.Debug().Zid(zettel.Meta.Zid).Msg("UpdateZettel")
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if mgr.State() != box.StartStateStarted {
	if !mgr.started {
		return box.ErrStopped
	}
	// Remove all (computed) properties from metadata before storing the zettel.
	zettel.Meta = zettel.Meta.Clone()
	for _, p := range zettel.Meta.ComputedPairsRest() {
		if mgr.propertyKeys.Has(p.Key) {
			zettel.Meta.Delete(p.Key)
		}
	}
	return mgr.boxes[0].UpdateZettel(ctx, zettel)
}

// AllowRenameZettel returns true, if box will not disallow renaming the zettel.
func (mgr *Manager) AllowRenameZettel(ctx context.Context, zid id.Zid) bool {
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if mgr.State() != box.StartStateStarted {
	if !mgr.started {
		return false
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	for _, p := range mgr.boxes {
		if !p.AllowRenameZettel(ctx, zid) {
			return false
		}
	}
	return true
}

// RenameZettel changes the current zid to a new zid.
func (mgr *Manager) RenameZettel(ctx context.Context, curZid, newZid id.Zid) error {
	mgr.mgrLog.Debug().Zid(curZid).Zid(newZid).Msg("RenameZettel")
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if mgr.State() != box.StartStateStarted {
	if !mgr.started {
		return box.ErrStopped
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	for i, p := range mgr.boxes {
		err := p.RenameZettel(ctx, curZid, newZid)
		if err != nil && !errors.Is(err, box.ErrNotFound) {
			for j := 0; j < i; j++ {
				mgr.boxes[j].RenameZettel(ctx, newZid, curZid)
			}
			return err
		}
	}
	return nil
}

// CanDeleteZettel returns true, if box could possibly delete the given zettel.
func (mgr *Manager) CanDeleteZettel(ctx context.Context, zid id.Zid) bool {
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if mgr.State() != box.StartStateStarted {
	if !mgr.started {
		return false
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	for _, p := range mgr.boxes {
		if p.CanDeleteZettel(ctx, zid) {
			return true
		}
	}
	return false
}

// DeleteZettel removes the zettel from the box.
func (mgr *Manager) DeleteZettel(ctx context.Context, zid id.Zid) error {
	mgr.mgrLog.Debug().Zid(zid).Msg("DeleteZettel")
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if mgr.State() != box.StartStateStarted {
	if !mgr.started {
		return box.ErrStopped
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	for _, p := range mgr.boxes {
		err := p.DeleteZettel(ctx, zid)
		if err == nil {
			return nil
		}
		if !errors.Is(err, box.ErrNotFound) && !errors.Is(err, box.ErrReadOnly) {
			return err
		}
	}
	return box.ErrNotFound
}

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package manager

import (
	"strings"

	"zettelstore.de/z/ast"
	"zettelstore.de/z/box/manager/store"
	"zettelstore.de/z/strfun"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/strfun"
)

type collectData struct {
	refs  id.Set
	words store.WordSet
	urls  store.WordSet
}

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package manager

import (
	"context"
	"strconv"

	"zettelstore.de/c/api"
	"zettelstore.de/z/box"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
)

// Enrich computes additional properties and updates the given metadata.
func (mgr *Manager) Enrich(ctx context.Context, m *meta.Meta, boxNumber int) {

	// Calculate computed, but stored values.
	if _, ok := m.Get(api.KeyCreated); !ok {

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package manager

import (
	"context"
	"fmt"
	"net/url"
	"time"

	"zettelstore.de/z/box"
	"zettelstore.de/z/box/manager/store"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/strfun"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

// SearchEqual returns all zettel that contains the given exact word.
// The word must be normalized through Unicode NKFD, trimmed and not empty.
func (mgr *Manager) SearchEqual(word string) id.Set {
	found := mgr.idxStore.SearchEqual(word)
	mgr.idxLog.Debug().Str("word", word).Int("found", int64(len(found))).Msg("SearchEqual")

		}
	}()

	timerDuration := 15 * time.Second
	timer := time.NewTimer(timerDuration)
	ctx := box.NoEnrichContext(context.Background())
	for {
		mgr.idxWorkService(ctx)
		// Sleep first, so the indexer will wait for boxes to initialize.
		if !mgr.idxSleepService(timer, timerDuration) {
			return
		}
		mgr.idxWorkService(ctx)
	}
}

func (mgr *Manager) idxWorkService(ctx context.Context) {
	var roomNum uint64
	var start time.Time
	for {

			<-timer.C
		}
		return false
	}
	return true
}

func (mgr *Manager) idxUpdateZettel(ctx context.Context, zettel zettel.Zettel) {
func (mgr *Manager) idxUpdateZettel(ctx context.Context, zettel domain.Zettel) {
	var cData collectData
	cData.initialize()
	collectZettelIndexData(parser.ParseZettel(ctx, zettel, "", mgr.rtConfig), &cData)

	m := zettel.Meta
	zi := store.NewZettelIndex(m.Zid)
	mgr.idxCollectFromMeta(ctx, m, zi, &cData)

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

	"zettelstore.de/c/maps"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/box"
	"zettelstore.de/z/box/manager/memstore"
	"zettelstore.de/z/box/manager/store"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/logger"
	"zettelstore.de/z/strfun"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

// ConnectData contains all administration related values.
type ConnectData struct {
	Number   int // number of the box, starting with 1.
	Config   config.Config
	Enricher box.Enricher

// GetSchemes returns all registered scheme, ordered by scheme string.
func GetSchemes() []string { return maps.Keys(registry) }

// Manager is a coordinating box.
type Manager struct {
	mgrLog       *logger.Logger
	stateMx      sync.RWMutex
	state        box.StartState
	mgrMx        sync.RWMutex
	started      bool
	rtConfig     config.Config
	boxes        []box.ManagedBox
	observers    []box.UpdateFunc
	mxObserver   sync.RWMutex
	done         chan struct{}
	infos        chan box.UpdateInfo
	propertyKeys strfun.Set // Set of property key names

	// Indexer stats data
	idxMx          sync.RWMutex
	idxLastReload  time.Time
	idxDurReload   time.Duration
	idxSinceReload uint64
}

func (mgr *Manager) setState(newState box.StartState) {
	mgr.stateMx.Lock()
	mgr.state = newState
	mgr.stateMx.Unlock()
}

func (mgr *Manager) State() box.StartState {
	mgr.stateMx.RLock()
	state := mgr.state
	mgr.stateMx.RUnlock()
	return state
}

// New creates a new managing box.
func New(boxURIs []*url.URL, authManager auth.BaseManager, rtConfig config.Config) (*Manager, error) {
	descrs := meta.GetSortedKeyDescriptions()
	propertyKeys := make(strfun.Set, len(descrs))
	for _, kd := range descrs {
		if kd.IsProperty() {
			propertyKeys.Set(kd.Name)

				reason, zid := ci.Reason, ci.Zid
				mgr.mgrLog.Debug().Uint("reason", uint64(reason)).Zid(zid).Msg("notifier")
				if ignoreUpdate(cache, now, reason, zid) {
					mgr.mgrLog.Trace().Uint("reason", uint64(reason)).Zid(zid).Msg("notifier ignored")
					continue
				}

				mgr.idxEnqueue(reason, zid)
				if ci.Box == nil {
					ci.Box = mgr
				}
				if mgr.State() == box.StartStateStarted {
					mgr.notifyObserver(&ci)
				mgr.notifyObserver(&ci)
				}
			}
		case <-mgr.done:
			return
		}
	}
}

		reason: reason,
	}
	return false
}

func (mgr *Manager) idxEnqueue(reason box.UpdateReason, zid id.Zid) {
	switch reason {
	case box.OnReady:
		return
	case box.OnReload:
		mgr.idxAr.Reset()
	case box.OnZettel:
		mgr.idxAr.EnqueueZettel(zid)
	default:
		mgr.mgrLog.Warn().Uint("reason", uint64(reason)).Zid(zid).Msg("Unknown notification reason")
		return
	}
	select {
	case mgr.idxReady <- struct{}{}:
	default:
	}
}

	}
}

// Start the box. Now all other functions of the box are allowed.
// Starting an already started box is not allowed.
func (mgr *Manager) Start(ctx context.Context) error {
	mgr.mgrMx.Lock()
	if mgr.started {
	defer mgr.mgrMx.Unlock()
		mgr.mgrMx.Unlock()
	if mgr.State() != box.StartStateStopped {
		return box.ErrStarted
	}
	mgr.setState(box.StartStateStarting)
	for i := len(mgr.boxes) - 1; i >= 0; i-- {
		ssi, ok := mgr.boxes[i].(box.StartStopper)
		if !ok {
			continue
		}
		err := ssi.Start(ctx)
		if err == nil {
			continue
		}
		mgr.setState(box.StartStateStopping)
		for j := i + 1; j < len(mgr.boxes); j++ {
			if ssj, ok2 := mgr.boxes[j].(box.StartStopper); ok2 {
				ssj.Stop(ctx)
			}
		}
		mgr.setState(box.StartStateStopped)
		mgr.mgrMx.Unlock()
		return err
	}
	mgr.idxAr.Reset() // Ensure an initial index run
	mgr.done = make(chan struct{})
	go mgr.notifier()

	mgr.waitBoxesAreStarted()
	mgr.setState(box.StartStateStarted)
	mgr.notifyObserver(&box.UpdateInfo{Box: mgr, Reason: box.OnReady})

	go mgr.idxIndexer()
	return nil
}


	mgr.started = true
func (mgr *Manager) waitBoxesAreStarted() {
	const waitTime = 10 * time.Millisecond
	const waitLoop = int(1 * time.Second / waitTime)
	for i := 1; !mgr.allBoxesStarted(); i++ {
		if i%waitLoop == 0 {
			if time.Duration(i)*waitTime > time.Minute {
				mgr.mgrLog.Warn().Msg("Waiting for more than one minute to start")
			} else {
				mgr.mgrLog.Trace().Msg("Wait for boxes to start")
			}
		}
		time.Sleep(waitTime)
	}
}

	mgr.mgrMx.Unlock()
func (mgr *Manager) allBoxesStarted() bool {
	for _, bx := range mgr.boxes {
		if b, ok := bx.(box.StartStopper); ok && b.State() != box.StartStateStarted {
			return false
	return nil
		}
	}
	return true
}

// Stop the started box. Now only the Start() function is allowed.
func (mgr *Manager) Stop(ctx context.Context) {
	mgr.mgrMx.Lock()
	defer mgr.mgrMx.Unlock()
	if mgr.State() != box.StartStateStarted {
	if !mgr.started {
		return
	}
	mgr.setState(box.StartStateStopping)
	close(mgr.done)
	for _, p := range mgr.boxes {
		if ss, ok := p.(box.StartStopper); ok {
			ss.Stop(ctx)
		}
	}
	mgr.setState(box.StartStateStopped)
	mgr.started = false
}

// Refresh internal box data.
func (mgr *Manager) Refresh(ctx context.Context) error {
	mgr.mgrLog.Debug().Msg("Refresh")
	mgr.mgrMx.Lock()
	defer mgr.mgrMx.Unlock()
	if mgr.State() != box.StartStateStarted {
	if !mgr.started {
		return box.ErrStopped
	}
	mgr.mgrMx.Lock()
	defer mgr.mgrMx.Unlock()
	mgr.infos <- box.UpdateInfo{Reason: box.OnReload, Zid: id.Invalid}
	for _, bx := range mgr.boxes {
		if rb, ok := bx.(box.Refresher); ok {
			rb.Refresh(ctx)
		}
	}
	return nil

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

	"sort"
	"strings"
	"sync"

	"zettelstore.de/c/api"
	"zettelstore.de/c/maps"
	"zettelstore.de/z/box/manager/store"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
)

type metaRefs struct {
	forward  id.Slice
	backward id.Slice
}

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package memstore

import "zettelstore.de/z/zettel/id"
import "zettelstore.de/z/domain/id"

func refsDiff(refsN, refsO id.Slice) (newRefs, remRefs id.Slice) {
	npos, opos := 0, 0
	for npos < len(refsN) && opos < len(refsO) {
		rn, ro := refsN[npos], refsO[opos]
		if rn == ro {
			npos++

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package memstore

import (
	"testing"

	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/domain/id"
)

func assertRefs(t *testing.T, i int, got, exp id.Slice) {
	t.Helper()
	if got == nil && exp != nil {
		t.Errorf("%d: got nil, but expected %v", i, exp)
		return

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package store contains general index data for storing a zettel index.
package store

import (
	"context"
	"io"

	"zettelstore.de/z/query"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/query"
)

// Stats records statistics about the store.
type Stats struct {
	// Zettel is the number of zettel managed by the indexer.
	Zettel int

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package store

import "zettelstore.de/z/zettel/id"
import "zettelstore.de/z/domain/id"

// ZettelIndex contains all index data of a zettel.
type ZettelIndex struct {
	Zid      id.Zid            // zid of the indexed zettel
	backrefs id.Set            // set of back references
	metarefs map[string]id.Set // references to inverse keys
	deadrefs id.Set            // set of dead references

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2023 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package membox stores zettel volatile in main memory.
package membox

import (
	"context"
	"net/url"
	"sync"

	"zettelstore.de/z/box"
	"zettelstore.de/z/box/manager"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/logger"
	"zettelstore.de/z/query"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func init() {
	manager.Register(
		"mem",
		func(u *url.URL, cdata *manager.ConnectData) (box.ManagedBox, error) {
			return &memBox{

type memBox struct {
	log       *logger.Logger
	u         *url.URL
	cdata     manager.ConnectData
	maxZettel int
	maxBytes  int
	mx        sync.RWMutex // Protects the following fields
	zettel    map[id.Zid]zettel.Zettel
	zettel    map[id.Zid]domain.Zettel
	curBytes  int
}

func (mb *memBox) notifyChanged(zid id.Zid) {
	if chci := mb.cdata.Notify; chci != nil {
		chci <- box.UpdateInfo{Box: mb, Reason: box.OnZettel, Zid: zid}
		chci <- box.UpdateInfo{Reason: box.OnZettel, Zid: zid}
	}
}

func (mb *memBox) Location() string {
	return mb.u.String()
}

func (mb *memBox) State() box.StartState {
	mb.mx.RLock()
	defer mb.mx.RUnlock()
	if mb.zettel == nil {
		return box.StartStateStopped
	}
	return box.StartStateStarted
}

func (mb *memBox) Start(context.Context) error {
	mb.mx.Lock()
	mb.zettel = make(map[id.Zid]zettel.Zettel)
	mb.zettel = make(map[id.Zid]domain.Zettel)
	mb.curBytes = 0
	mb.mx.Unlock()
	mb.log.Trace().Int("max-zettel", int64(mb.maxZettel)).Int("max-bytes", int64(mb.maxBytes)).Msg("Start Box")
	return nil
}

func (mb *memBox) Stop(context.Context) {
	mb.mx.Lock()
	mb.zettel = nil
	mb.mx.Unlock()
}

func (mb *memBox) CanCreateZettel(context.Context) bool {
	mb.mx.RLock()
	defer mb.mx.RUnlock()
	return len(mb.zettel) < mb.maxZettel
}

func (mb *memBox) CreateZettel(_ context.Context, zettel zettel.Zettel) (id.Zid, error) {
func (mb *memBox) CreateZettel(_ context.Context, zettel domain.Zettel) (id.Zid, error) {
	mb.mx.Lock()
	newBytes := mb.curBytes + zettel.Length()
	if mb.maxZettel < len(mb.zettel) || mb.maxBytes < newBytes {
		mb.mx.Unlock()
		return id.Invalid, box.ErrCapacity
	}
	zid, err := box.GetNewZid(func(zid id.Zid) (bool, error) {

	mb.curBytes = newBytes
	mb.mx.Unlock()
	mb.notifyChanged(zid)
	mb.log.Trace().Zid(zid).Msg("CreateZettel")
	return zid, nil
}

func (mb *memBox) GetZettel(_ context.Context, zid id.Zid) (zettel.Zettel, error) {
func (mb *memBox) GetZettel(_ context.Context, zid id.Zid) (domain.Zettel, error) {
	mb.mx.RLock()
	z, ok := mb.zettel[zid]
	zettel, ok := mb.zettel[zid]
	mb.mx.RUnlock()
	if !ok {
		return zettel.Zettel{}, box.ErrNotFound
		return domain.Zettel{}, box.ErrNotFound
	}
	z.Meta = z.Meta.Clone()
	zettel.Meta = zettel.Meta.Clone()
	mb.log.Trace().Msg("GetZettel")
	return z, nil
	return zettel, nil
}

func (mb *memBox) GetMeta(_ context.Context, zid id.Zid) (*meta.Meta, error) {
	mb.mx.RLock()
	zettel, ok := mb.zettel[zid]
	mb.mx.RUnlock()
	if !ok {

			mb.cdata.Enricher.Enrich(ctx, m, mb.cdata.Number)
			handle(m)
		}
	}
	return nil
}

func (mb *memBox) CanUpdateZettel(_ context.Context, zettel zettel.Zettel) bool {
func (mb *memBox) CanUpdateZettel(_ context.Context, zettel domain.Zettel) bool {
	mb.mx.RLock()
	defer mb.mx.RUnlock()
	zid := zettel.Meta.Zid
	if !zid.IsValid() {
		return false
	}

	newBytes := mb.curBytes + zettel.Length()
	if prevZettel, found := mb.zettel[zid]; found {
		newBytes -= prevZettel.Length()
	}
	return newBytes < mb.maxBytes
}

func (mb *memBox) UpdateZettel(_ context.Context, zettel zettel.Zettel) error {
func (mb *memBox) UpdateZettel(_ context.Context, zettel domain.Zettel) error {
	m := zettel.Meta.Clone()
	if !m.Zid.IsValid() {
		return &box.ErrInvalidID{Zid: m.Zid}
	}

	mb.mx.Lock()
	newBytes := mb.curBytes + zettel.Length()

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2023 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package notify

import (
	"errors"
	"fmt"
	"path/filepath"
	"regexp"
	"strings"
	"sync"

	"zettelstore.de/z/box"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/logger"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/query"
	"zettelstore.de/z/strfun"
	"zettelstore.de/z/zettel/id"
)

type entrySet map[id.Zid]*DirEntry

// DirServiceState signal the internal state of the service.
// directoryState signal the internal state of the service.
//
// The following state transitions are possible:
// --newDirService--> dsCreated
// dsCreated --Start--> dsStarting
// dsStarting --last list notification--> dsWorking
// dsWorking --directory missing--> dsMissing
// dsMissing --last list notification--> dsWorking
// --Stop--> dsStopping
type DirServiceState uint8
type directoryState uint8

const (
	DsCreated  DirServiceState = iota
	DsStarting                 // Reading inital scan
	DsWorking                  // Initial scan complete, fully operational
	DsMissing                  // Directory is missing
	DsStopping                 // Service is shut down
	dsCreated  directoryState = iota
	dsStarting                // Reading inital scan
	dsWorking                 // Initial scan complete, fully operational
	dsMissing                 // Directory is missing
	dsStopping                // Service is shut down
)

// DirService specifies a directory service for file based zettel.
type DirService struct {
	box      box.ManagedBox
	log      *logger.Logger
	dirPath  string
	notifier Notifier
	infos    chan<- box.UpdateInfo
	mx       sync.RWMutex // protects status, entries
	state    DirServiceState
	state    directoryState
	entries  entrySet
}

// ErrNoDirectory signals missing directory data.
var ErrNoDirectory = errors.New("unable to retrieve zettel directory information")

// NewDirService creates a new directory service.
func NewDirService(box box.ManagedBox, log *logger.Logger, notifier Notifier, chci chan<- box.UpdateInfo) *DirService {
func NewDirService(log *logger.Logger, notifier Notifier, chci chan<- box.UpdateInfo) *DirService {
	return &DirService{
		box:      box,
		log:      log,
		notifier: notifier,
		infos:    chci,
		state:    DsCreated,
		state:    dsCreated,
	}
}

// State the current service state.
func (ds *DirService) State() DirServiceState {
	ds.mx.RLock()
	state := ds.state
	ds.mx.RUnlock()
	return state
}

// Start the directory service.
func (ds *DirService) Start() {
	ds.mx.Lock()
	ds.state = DsStarting
	ds.state = dsStarting
	ds.mx.Unlock()
	var newEntries entrySet
	go ds.updateEvents(newEntries)
}

// Refresh the directory entries.
func (ds *DirService) Refresh() {
	ds.notifier.Refresh()
}

// Stop the directory service.
func (ds *DirService) Stop() {
	ds.mx.Lock()
	ds.state = DsStopping
	ds.state = dsStopping
	ds.mx.Unlock()
	ds.notifier.Close()
}

func (ds *DirService) logMissingEntry(action string) error {
	err := ErrNoDirectory
	ds.log.Info().Err(err).Str("action", action).Msg("Unable to get directory information")

	ds.mx.RLock()
	state := ds.state
	ds.mx.RUnlock()

	if msg := ds.log.Trace(); msg.Enabled() {
		msg.Uint("state", uint64(state)).Str("op", ev.Op.String()).Str("name", ev.Name).Msg("notifyEvent")
	}
	if state == DsStopping {
	if state == dsStopping {
		return nil, false
	}

	switch ev.Op {
	case Error:
		newEntries = nil
		if state != DsMissing {
		if state != dsMissing {
			ds.log.Warn().Err(ev.Err).Msg("Notifier confused")
		}
	case Make:
		newEntries = make(entrySet)
	case List:
		if ev.Name == "" {
			zids := getNewZids(newEntries)
			ds.mx.Lock()
			fromMissing := ds.state == DsMissing
			fromMissing := ds.state == dsMissing
			prevEntries := ds.entries
			ds.entries = newEntries
			ds.state = DsWorking
			ds.state = dsWorking
			ds.mx.Unlock()
			ds.onCreateDirectory(zids, prevEntries)
			if fromMissing {
				ds.log.Info().Str("path", ds.dirPath).Msg("Zettel directory found")
			}
			return nil, true
		}

	}
}

func (ds *DirService) onDestroyDirectory() {
	ds.mx.Lock()
	entries := ds.entries
	ds.entries = nil
	ds.state = DsMissing
	ds.state = dsMissing
	ds.mx.Unlock()
	for zid := range entries {
		ds.notifyChange(zid)
	}
}

var validFileName = regexp.MustCompile(`^(\d{14})`)

	}
	return newExt < oldExt
}

func (ds *DirService) notifyChange(zid id.Zid) {
	if chci := ds.infos; chci != nil {
		ds.log.Trace().Zid(zid).Msg("notifyChange")
		chci <- box.UpdateInfo{Box: ds.box, Reason: box.OnZettel, Zid: zid}
		chci <- box.UpdateInfo{Reason: box.OnZettel, Zid: zid}
	}
}

//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
// Copyright (c) 2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package notify

import (
	"testing"

	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	_ "zettelstore.de/z/parser/blob"       // Allow to use BLOB parser.
	_ "zettelstore.de/z/parser/draw"       // Allow to use draw parser.
	_ "zettelstore.de/z/parser/markdown"   // Allow to use markdown parser.
	_ "zettelstore.de/z/parser/none"       // Allow to use none parser.
	_ "zettelstore.de/z/parser/plain"      // Allow to use plain parser.
	_ "zettelstore.de/z/parser/zettelmark" // Allow to use zettelmark parser.
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func TestSeekZid(t *testing.T) {
	testcases := []struct {
		name string
		zid  id.Zid
	}{

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package notify

import (
	"path/filepath"

	"zettelstore.de/c/api"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/parser"
)

const (
	extZettel = "zettel" // file contains metadata and content
	extBin    = "bin"    // file contains binary content
	extTxt    = "txt"    // file contains non-binary content
)

// HasMetaInContent returns true, if metadata will be stored in the content file.
func (e *DirEntry) HasMetaInContent() bool {
	return e.IsValid() && extIsMetaAndContent(e.ContentExt)
}

// SetupFromMetaContent fills entry data based on metadata and zettel content.
func (e *DirEntry) SetupFromMetaContent(m *meta.Meta, content zettel.Content, getZettelFileSyntax func() []string) {
func (e *DirEntry) SetupFromMetaContent(m *meta.Meta, content domain.Content, getZettelFileSyntax func() []string) {
	if e.Zid != m.Zid {
		panic("Zid differ")
	}
	if contentName := e.ContentName; contentName != "" {
		if !extIsMetaAndContent(e.ContentExt) && e.MetaName == "" {
			e.MetaName = e.calcBaseName(contentName)
		}

		}
		if metaName == "" {
			e.MetaName = e.calcBaseName(e.ContentName)
		}
	}
}

func contentExtWithMeta(syntax string, content zettel.Content) string {
func contentExtWithMeta(syntax string, content domain.Content) string {
	p := parser.Get(syntax)
	if content.IsBinary() {
		if p.IsImageFormat {
			return syntax
		}
		return extBin
	}

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

func (fsdn *fsdirNotifier) eventLoop() {
	defer fsdn.base.Close()
	defer close(fsdn.events)
	defer close(fsdn.refresh)
	if !listDirElements(fsdn.log, fsdn.fetcher, fsdn.events, fsdn.done) {
		return
	}

	for fsdn.readAndProcessEvent() {
	}
}

func (fsdn *fsdirNotifier) readAndProcessEvent() bool {
	select {
	case <-fsdn.done:

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2023 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package cmd

import (
	"context"
	"flag"
	"fmt"
	"io"
	"os"

	"zettelstore.de/c/api"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

// ---------- Subcommand: file -----------------------------------------------

func cmdFile(fs *flag.FlagSet) (int, error) {
	enc := fs.Lookup("t").Value.String()
	m, inp, err := getInput(fs.Args())
	if m == nil {
		return 2, err
	}
	z := parser.ParseZettel(
		context.Background(),
		zettel.Zettel{
		domain.Zettel{
			Meta:    m,
			Content: zettel.NewContent(inp.Src[inp.Pos:]),
			Content: domain.NewContent(inp.Src[inp.Pos:]),
		},
		m.GetDefault(api.KeySyntax, meta.SyntaxZmk),
		nil,
	)
	encdr := encoder.Create(api.Encoder(enc))
	if encdr == nil {
		fmt.Fprintf(os.Stderr, "Unknown format %q\n", enc)

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package cmd

import (
	"flag"
	"fmt"
	"os"

	"golang.org/x/term"

	"zettelstore.de/c/api"
	"zettelstore.de/z/auth/cred"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/domain/id"
)

// ---------- Subcommand: password -------------------------------------------

func cmdPassword(fs *flag.FlagSet) (int, error) {
	if fs.NArg() == 0 {
		fmt.Fprintln(os.Stderr, "User name and user zettel identification missing")

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2023 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package cmd

import (
	"context"
	"flag"
	"net/http"

	"zettelstore.de/z/auth"
	"zettelstore.de/z/box"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter/api"
	"zettelstore.de/z/web/adapter/webui"
	"zettelstore.de/z/web/server"
	"zettelstore.de/z/zettel/meta"
)

// ---------- Subcommand: run ------------------------------------------------

func flgRun(fs *flag.FlagSet) {
	fs.String("c", "", "configuration file")
	fs.Uint("a", 0, "port number kernel service (0=disable)")

	ucGetAllMeta := usecase.NewGetAllMeta(protectedBoxManager)
	ucGetZettel := usecase.NewGetZettel(protectedBoxManager)
	ucParseZettel := usecase.NewParseZettel(rtConfig, ucGetZettel)
	ucListMeta := usecase.NewListMeta(protectedBoxManager)
	ucEvaluate := usecase.NewEvaluate(rtConfig, ucGetZettel, ucGetMeta, ucListMeta)
	ucListSyntax := usecase.NewListSyntax(protectedBoxManager)
	ucListRoles := usecase.NewListRoles(protectedBoxManager)
	ucZettelContext := usecase.NewZettelContext(protectedBoxManager, rtConfig)
	ucDelete := usecase.NewDeleteZettel(logUc, protectedBoxManager)
	ucUpdate := usecase.NewUpdateZettel(logUc, protectedBoxManager)
	ucRename := usecase.NewRenameZettel(logUc, protectedBoxManager)
	ucUnlinkedRefs := usecase.NewUnlinkedReferences(protectedBoxManager, rtConfig)
	ucRefresh := usecase.NewRefresh(logUc, protectedBoxManager)
	ucVersion := usecase.NewVersion(kernel.Main.GetConfig(kernel.CoreService, kernel.CoreVersion).(string))

		const assetPrefix = "/assets/"
		webSrv.Handle(assetPrefix, http.StripPrefix(assetPrefix, http.FileServer(http.Dir(assetDir))))
		webSrv.Handle("/favicon.ico", wui.MakeFaviconHandler(assetDir))
	}

	// Web user interface
	if !authManager.IsReadonly() {
		webSrv.AddZettelRoute('b', server.MethodGet, wui.MakeGetRenameZettelHandler(ucGetMeta))
		webSrv.AddZettelRoute('b', server.MethodGet, wui.MakeGetRenameZettelHandler(
			ucGetMeta, &ucEvaluate))
		webSrv.AddZettelRoute('b', server.MethodPost, wui.MakePostRenameZettelHandler(&ucRename))
		webSrv.AddListRoute('c', server.MethodGet, wui.MakeGetZettelFromListHandler(ucListMeta, &ucEvaluate, ucListRoles, ucListSyntax))
		webSrv.AddListRoute('c', server.MethodPost, wui.MakePostCreateZettelHandler(&ucCreateZettel))
		webSrv.AddZettelRoute('c', server.MethodGet, wui.MakeGetCreateZettelHandler(
			ucGetZettel, &ucCreateZettel, ucListRoles, ucListSyntax))
		webSrv.AddZettelRoute('c', server.MethodPost, wui.MakePostCreateZettelHandler(&ucCreateZettel))
		webSrv.AddZettelRoute('d', server.MethodGet, wui.MakeGetDeleteZettelHandler(ucGetMeta, ucGetAllMeta))
		webSrv.AddZettelRoute('d', server.MethodGet, wui.MakeGetDeleteZettelHandler(
			ucGetMeta, ucGetAllMeta, &ucEvaluate))
		webSrv.AddZettelRoute('d', server.MethodPost, wui.MakePostDeleteZettelHandler(&ucDelete))
		webSrv.AddZettelRoute('e', server.MethodGet, wui.MakeEditGetZettelHandler(ucGetZettel, ucListRoles, ucListSyntax))
		webSrv.AddZettelRoute('e', server.MethodPost, wui.MakeEditSetZettelHandler(&ucUpdate))
	}
	webSrv.AddListRoute('g', server.MethodGet, wui.MakeGetGoActionHandler(&ucRefresh))
	webSrv.AddListRoute('h', server.MethodGet, wui.MakeListHTMLMetaHandler(ucListMeta))
	webSrv.AddListRoute('h', server.MethodGet, wui.MakeListHTMLMetaHandler(ucListMeta, &ucEvaluate))
	webSrv.AddZettelRoute('h', server.MethodGet, wui.MakeGetHTMLZettelHandler(&ucEvaluate, ucGetMeta))
	webSrv.AddListRoute('i', server.MethodGet, wui.MakeGetLoginOutHandler())
	webSrv.AddListRoute('i', server.MethodPost, wui.MakePostLoginHandler(&ucAuthenticate))
	webSrv.AddZettelRoute('i', server.MethodGet, wui.MakeGetInfoHandler(
		ucParseZettel, &ucEvaluate, ucGetMeta, ucGetAllMeta, ucUnlinkedRefs))
	webSrv.AddZettelRoute('k', server.MethodGet, wui.MakeZettelContextHandler(
		ucZettelContext, &ucEvaluate))

	// API
	webSrv.AddListRoute('a', server.MethodPost, a.MakePostLoginHandler(&ucAuthenticate))
	webSrv.AddListRoute('a', server.MethodPut, a.MakeRenewAuthHandler())
	webSrv.AddZettelRoute('o', server.MethodGet, a.MakeGetOrderHandler(
		usecase.NewZettelOrder(protectedBoxManager, ucEvaluate)))
	webSrv.AddZettelRoute('u', server.MethodGet, a.MakeListUnlinkedMetaHandler(ucGetMeta, ucUnlinkedRefs))
	webSrv.AddZettelRoute('u', server.MethodGet, a.MakeListUnlinkedMetaHandler(
		ucGetMeta, ucUnlinkedRefs, &ucEvaluate))
	webSrv.AddListRoute('x', server.MethodGet, a.MakeGetDataHandler(ucVersion))
	webSrv.AddListRoute('x', server.MethodPost, a.MakePostCommandHandler(&ucIsAuth, &ucRefresh))
	webSrv.AddZettelRoute('x', server.MethodGet, a.MakeZettelContextHandler(ucZettelContext))
	webSrv.AddListRoute('z', server.MethodGet, a.MakeQueryHandler(ucListMeta))
	webSrv.AddZettelRoute('z', server.MethodGet, a.MakeGetZettelHandler(ucGetMeta, ucGetZettel, ucParseZettel, ucEvaluate))
	if !authManager.IsReadonly() {
		webSrv.AddListRoute('z', server.MethodPost, a.MakePostCreateZettelHandler(&ucCreateZettel))
		webSrv.AddZettelRoute('z', server.MethodPut, a.MakeUpdateZettelHandler(&ucUpdate))
		webSrv.AddZettelRoute('z', server.MethodDelete, a.MakeDeleteZettelHandler(&ucDelete))
		webSrv.AddZettelRoute('z', server.MethodMove, a.MakeRenameZettelHandler(&ucRename))

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

	"zettelstore.de/c/api"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/auth/impl"
	"zettelstore.de/z/box"
	"zettelstore.de/z/box/compbox"
	"zettelstore.de/z/box/manager"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/input"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/logger"
	"zettelstore.de/z/web/server"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

const strRunSimple = "run-simple"

func init() {
	RegisterCommand(Command{
		Name: "help",

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package cmd provides command generic functions.
package cmd

// Mention all needed encoders, parsers and stores to have them registered.
import (
	_ "zettelstore.de/z/box/compbox"       // Allow to use computed box.
	_ "zettelstore.de/z/box/constbox"      // Allow to use global internal box.
	_ "zettelstore.de/z/box/dirbox"        // Allow to use directory box.
	_ "zettelstore.de/z/box/filebox"       // Allow to use file box.
	_ "zettelstore.de/z/box/membox"        // Allow to use in-memory box.
	_ "zettelstore.de/z/encoder/htmlenc"   // Allow to use HTML encoder.
	_ "zettelstore.de/z/encoder/mdenc"     // Allow to use markdown encoder.
	_ "zettelstore.de/z/encoder/shtmlenc"  // Allow to use SHTML encoder.
	_ "zettelstore.de/z/encoder/szenc"     // Allow to use Sz encoder.
	_ "zettelstore.de/z/encoder/sexprenc"  // Allow to use sexpr encoder.
	_ "zettelstore.de/z/encoder/textenc"   // Allow to use text encoder.
	_ "zettelstore.de/z/encoder/zjsonenc"  // Allow to use ZJSON encoder.
	_ "zettelstore.de/z/encoder/zmkenc"    // Allow to use zmk encoder.
	_ "zettelstore.de/z/kernel/impl"       // Allow kernel implementation to create itself
	_ "zettelstore.de/z/parser/blob"       // Allow to use BLOB parser.
	_ "zettelstore.de/z/parser/draw"       // Allow to use draw parser.
	_ "zettelstore.de/z/parser/markdown"   // Allow to use markdown parser.
	_ "zettelstore.de/z/parser/none"       // Allow to use none parser.
	_ "zettelstore.de/z/parser/plain"      // Allow to use plain parser.
	_ "zettelstore.de/z/parser/zettelmark" // Allow to use zettelmark parser.
)

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package config provides functions to retrieve runtime configuration data.
package config

import (
	"context"

	"zettelstore.de/z/zettel/meta"
	"zettelstore.de/z/domain/meta"
)

// Key values that are supported by Config.Get
const (
	KeyFooterZettel = "footer-zettel"
	KeyHomeZettel   = "home-zettel"
	// api.KeyLang
	KeyMarkerExternal = "marker-external"
)

// Config allows to retrieve all defined configuration values that can be changed during runtime.
type Config interface {
	AuthConfig

	// Get returns the value of the given key. It searches first in the given metadata,

id: 20210916193200
title: Required Software
role: zettel
syntax: zmk
created: 20210916193200
modified: 20230405150541
modified: 20230109121417

The following software must be installed:

* A current, supported [[release of Go|https://go.dev/doc/devel/release]],
* [[shadow|https://pkg.go.dev/golang.org/x/tools/go/analysis/passes/shadow]] via ``go install golang.org/x/tools/go/analysis/passes/shadow/cmd/shadow@latest``,
* [[staticcheck|https://staticcheck.io/]] via ``go install honnef.co/go/tools/cmd/staticcheck@latest``,
* [[unparam|https://mvdan.cc/unparam]][^[[GitHub|https://github.com/mvdan/unparam]]] via ``go install mvdan.cc/unparam@latest``,
* [[Fossil|https://fossil-scm.org/]],
* [[Git|https://git-scm.org/]] (most dependencies are accessible via Git only).

Make sure that the software is in your path, e.g. via:

```sh
export PATH=$PATH:/usr/local/go/bin
export PATH=$PATH:$(go env GOPATH)/bin
```

The internal build tool need the following software.
It can be installed / updated via the build tool itself: ``go run tools/build.go tools``.

Otherwise you can install the software by hand:

* [[shadow|https://pkg.go.dev/golang.org/x/tools/go/analysis/passes/shadow]] via ``go install golang.org/x/tools/go/analysis/passes/shadow/cmd/shadow@latest``,
* [[staticcheck|https://staticcheck.io/]] via ``go install honnef.co/go/tools/cmd/staticcheck@latest``,
* [[unparam|https://mvdan.cc/unparam]][^[[GitHub|https://github.com/mvdan/unparam]]] via ``go install mvdan.cc/unparam@latest``,
* [[govulncheck|https://golang.org/x/vuln/cmd/govulncheck]] via ``go install golang.org/x/vuln/cmd/govulncheck@latest``,

id: 20210916194900
title: Checklist for Release
role: zettel
syntax: zmk
created: 20210916194900
modified: 20230402181229
modified: 20220309105459

# Sync with the official repository
#* ``fossil sync -u``
# Make sure that there is no workspace defined.
#* ``ls ..`` must not have a file ''go.work'', in no parent folder.
# Make sure that all dependencies are up-to-date.
#* ``cat go.mod``

# Clean up your Go workspace:
#* ``go run tools/build.go clean`` (alternatively: ``make clean``).
# Create the release:
#* ``go run tools/build.go release`` (alternatively: ``make release``).
# Remove previous executables:
#* ``fossil uv remove --glob '*-PREVVERSION*'``
# Add executables for release:
#* ``cd releases``
#* ``cd release``
#* ``fossil uv add *.zip``
#* ``cd ..``
#* Synchronize with main repository:
#* ``fossil sync -u``
# Enable autosync:
#* ``fossil setting autosync on``

id: 00000000000100
title: Zettelstore Runtime Configuration
role: configuration
syntax: none
created: 00010101000000
default-copyright: (c) 2020-present by Detlef Stern <ds@zettelstore.de>
default-copyright: (c) 2020-2022 by Detlef Stern <ds@zettelstore.de>
default-license: EUPL-1.2-or-later
default-visibility: public
footer-zettel: 00001000000100
home-zettel: 00001000000000
modified: 20221205173642
site-name: Zettelstore Manual
visibility: owner

id: 00001004020000
title: Configure the running Zettelstore
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20230317183435
modified: 20221219175720

You can configure a running Zettelstore by modifying the special zettel with the ID [[00000000000100]].
This zettel is called __configuration zettel__.
The following metadata keys change the appearance / behavior of Zettelstore.
Some of them can be overwritten in an [[user zettel|00001010040200]], a subset of those may be overwritten in zettel that is currently used.
See the full list of [[metadata that may be overwritten|00001004020200]].

  Default: ""en"".

  This value is used as a default value, if it is not set in an user's zettel or in a zettel.
  It is also used to specify the language for all non-zettel content, e.g. lists or search results.

  Use values according to the language definition of [[RFC-5646|https://tools.ietf.org/html/rfc5646]].
; [!marker-external|''marker-external'']
: Some HTML code that is displayed after a [[reference to external material|00001007040310]].

  May be [[overwritten|00001004020200]] in a user zettel.

  Default: ""&\#10138;"", to display a ""➚"" sign.
; [!max-transclusions|''max-transclusions'']
: Maximum number of indirect transclusion.
  This is used to avoid an exploding ""transclusion bomb"", a form of a [[billion laughs attack|https://en.wikipedia.org/wiki/Billion_laughs_attack]].
  Default: ""1024"".
; [!site-name|''site-name'']
: Name of the Zettelstore instance.
  Will be used when displaying some lists.

id: 00001004020200
title: Runtime configuration data that may be user specific or zettel specific
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk
created: 20221205155521
modified: 20230317183403
modified: 20221212195530

Some metadata of the [[runtime configuration|00001004020000]] may be overwritten in an [[user zettel|00001010040200]].
A subset of those may be overwritten in zettel that is currently used.
This allows to specify user specific or zettel specific behavior.

The following metadata keys are supported to provide a more specific behavior:

|=Key|User:|Zettel:|Remarks
|[[''footer-zettel''|00001004020000#footer-zettel]]|Y|N|
|[[''home-zettel''|00001004020000#home-zettel]]|Y|N|
|[[''lang''|00001004020000#lang]]|Y|Y|Making it user-specific could make zettel for other user less useful
|[[''marker-external''|00001004020000#marker-external]]|Y|Y|

id: 00001004051200
title: The ''file'' sub-command
role: manual
tags: #command #configuration #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20230316182711
modified: 20230109105434

Reads zettel data from a file (or from standard input / stdin) and renders it to standard output / stdout.
This allows Zettelstore to render files manually.
```
zettelstore file [-t FORMAT] [file-1 [file-2]]
```

; ''-t FORMAT''
: Specifies the output format.
  Supported values are:
  [[''html''|00001012920510]] (default),
  [[''md''|00001012920513]],
  [[''shtml''|00001012920525]],
  [[''sz''|00001012920516]],
  [[''sexpr''|00001012920516]],
  [[''text''|00001012920519]],
  [[''zjson''|00001012920503]] (deprecated in v0.11),
  and [[''zmk''|00001012920522]].
; ''file-1''
: Specifies the file name, where at least metadata is read.
  If ''file-2'' is not given, the zettel content is also read from here.
; ''file-2''
: File name where the zettel content is stored.

If neither ''file-1'' nor ''file-2'' are given, metadata and zettel content are read from standard input / stdin.

id: 00001006000000
title: Layout of a Zettel
role: manual
tags: #design #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20230403123541
modified: 20221219162703

A zettel consists of two parts: the metadata and the zettel content.
Metadata gives some information mostly about the zettel content, how it should be interpreted, how it is sorted within Zettelstore.
The zettel content is, well, the actual content.
In many cases, the content is in plain text form.
Plain text is long-lasting.
However, content in binary format is also possible.

Other character encodings are not supported and will never be[^This is not a real problem, since every modern software should support UTF-8 as an encoding.].
There is support for a graphical format with a text representation: SVG.
And there is support for some binary image formats, like GIF, PNG, and JPEG.

=== Plain, parsed, and evaluated zettel
Zettelstore may present your zettel in various forms, typically retrieved with the [[endpoint|00001012920000]] ''/z/{ID}''.
One way is to present the zettel as it was read by Zettelstore.
This is called ""[[plain zettel|00001012053300]]"".
This is called ""[[plain zettel|00001003000000]]"".

The second way is to present the zettel as it was recognized by Zettelstore.
This is called ""[[parsed zettel|00001012053600]]"", also retrieved with the [[endpoint|00001012920000]] ''/z/{ID}'', but with the additional query parameter ''parseonly''.
This is called ""[[parsed zettel|00001012053600]]"", also retrieved with the [[endpoint|00001012920000]] ''/z/{ID}'', but with additional query parameter.
Such a zettel was read and analyzed.
It can be presented in various [[encodings|00001012920500]].[^The [[zmk encoding|00001012920522]] allows you to compare the plain, the parsed, and the evaluated form of a zettel.]

However, a zettel such as this one you are currently reading, is a ""[[evaluated zettel|00001012053500]]"", also retrieved with the [[endpoint|00001012920000]] ''/z/{ID}'' and specifying an encoding.
The biggest difference to a parsed zettel is the inclusion of [[block transclusions|00001007031100]] or [[inline transclusions|00001007040324]] for an evaluated zettel.
It can also be presented in various encoding, including the ""zmk"" encoding.
Evaluations also applies to metadata of a zettel, if appropriate.

Please note, that searching for content is based on parsed zettel.
Transcluded content will only be found in transcluded zettel, but not in the zettel that transcluded the content.
However, you will easily pick up that zettel by follow the [[backward|00001006020000#backward]] metadata key of the transcluded zettel.

id: 00001006020000
title: Supported Metadata Keys
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
created: 20210126175322
modified: 20230421155051
modified: 20221004134841

Although you are free to define your own metadata, by using any key (according to the [[syntax|00001006010000]]), some keys have a special meaning that is enforced by Zettelstore.
See the [[computed list of supported metadata keys|00000000000090]] for details.

Most keys conform to a [[type|00001006030000]].

; [!author|''author'']

; [!credential|''credential'']
: Contains the hashed password, as it was emitted by [[``zettelstore password``|00001004051400]].
  It is internally created by hashing the password, the [[zettel identifier|00001006050000]], and the value of the ''ident'' key.

  It is only used for zettel with a ''role'' value of ""user"".
; [!dead|''dead'']
: Property that contains all references that does __not__ identify a zettel.
; [!expire|''expire'']
: A user-entered time stamp that document the point in time when the zettel should expire.
  When a zettel is expires, Zettelstore does nothing.
  It is up to you to define required actions.
  ''expire'' is just a documentation.
  You could define a query and execute it regularly, for example [[query:expire? ORDER expire]].
  Alternatively, a Zettelstore client software could define some actions when it detects expired zettel.
; [!folge|''folge'']
: Is a property that contains identifier of all zettel that reference this zettel through the [[''precursor''|#precursor]] value.
; [!forward|''forward'']
: Property that contains all references that identify another zettel within the content of the zettel.
; [!id|''id'']
: Contains the [[zettel identifier|00001006050000]], as given by the Zettelstore.
  It cannot be set manually, because it is a computed value.

  Otherwise, if the zettel identifier contains a valid timestamp, the identifier is used.
  In all other cases, this property is not set.

  It can be used for [[sorting|00001007700000]] zettel based on their publication date.

  It is a computed value.
  There is no need to set it via Zettelstore.
; [!query|''query'']
: Stores the [[query|00001007031140]] that was used to create the zettel.
  This is for future reference.
; [!read-only|''read-only'']
: Marks a zettel as read-only.
  The interpretation of [[supported values|00001006020400]] for this key depends, whether authentication is [[enabled|00001010040100]] or not.
; [!role|''role'']
: Defines the role of the zettel.
  Can be used for selecting zettel.
  See [[supported zettel roles|00001006020100]].
  If not given, it is ignored.
; [!subordinates|''subordinates'']
: Is a property that contains identifier of all zettel that reference this zettel through the [[''superior''|#superior]] value.
; [!successors|''successors'']
: Is a property that contains identifier of all zettel that reference this zettel through the [[''predecessor''|#predecessor]] value.
  Therefore, it references all zettel that contain a new version of the content and/or metadata.
  In contrast to [[''folge''|#folge]], these are references because of technical reasons, not because of content-related reasons.
  In most cases, zettel referencing the current zettel should be updated to reference a successor zettel.
  The [[query reference|00001007040310]] [[query:backward? successors?]] lists all such zettel.
; [!summary|''summary'']
: Summarizes the content of the zettel.
  You may use all [[inline-structued elements|00001007040000]] of Zettelmarkup.
; [!superior|''superior'']
: Specifies a zettel that is conceptually a superior zettel.
  This might be a more abstract zettel, or a zettel that should be higher in a hierarchy.
; [!syntax|''syntax'']
: Specifies the syntax that should be used for interpreting the zettel.
  The zettel about [[other markup languages|00001008000000]] defines supported values.
  If it is not given, it defaults to ''plain''.
; [!tags|''tags'']
: Contains a space separated list of tags to describe the zettel further.
  Each Tag must begin with the number sign character (""''#''"", U+0023).
; [!title|''title'']
: Specifies the title of the zettel.
  If not given, the value of [[''id''|#id]] will be used.

  You can use all [[inline-structured elements|00001007040000]] of Zettelmarkup.
; [!url|''url'']
: Defines an URL / URI for this zettel that possibly references external material.
  One use case is to specify the document that the current zettel comments on.
  The URL will be rendered special in the [[web user interface|00001014000000]] if you use the default template.
; [!useless-files|''useless-files'']
: Contains the file names that are rejected to serve the content of a zettel.
  Is used for [[directory boxes|00001004011400]] and [[file boxes|00001004011200#file]].

id: 00001006030000
title: Supported Key Types
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
created: 20210126175322
modified: 20230402183536
modified: 20220304114106

All [[supported metadata keys|00001006020000]] conform to a type.

User-defined metadata keys conform also to a type, based on the suffix of the key.

|=Suffix|Type
| ''-date'' | [[Timestamp|00001006034500]]
| ''-number'' | [[Number|00001006033000]]
| ''-role'' | [[Word|00001006035500]]
| ''-set'' | [[WordSet|00001006036000]]
| ''-time'' | [[Timestamp|00001006034500]]
| ''-title'' | [[Zettelmarkup|00001006036500]]
| ''-url'' | [[URL|00001006035000]]
| ''-zettel''  | [[Identifier|00001006032000]]
| ''-zid''  | [[Identifier|00001006032000]]
| ''-zids''  | [[IdentifierSet|00001006032500]]
| any other suffix | [[EString|00001006031500]]

id: 00001006031000
title: Credential Key Type
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
created: 20210212135017
modified: 20230419175515
modified: 20220914130324

Values of this type denote a credential value, e.g. an encrypted password.

=== Allowed values
All printable characters are allowed.
Since a credential contains some kind of secret, the sequence of characters might have some hidden syntax to be interpreted by other parts of Zettelstore.

=== Query comparison
=== Query operators
A credential never compares to any other value.
A comparison will never match in any way.

=== Sorting
If a list of zettel should be sorted based on a credential value, the identifier of the respective zettel is used instead.

id: 00001006031500
title: EString Key Type
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
created: 20210212135017
modified: 20230419175525
modified: 20220914130448

Values of this type are just a sequence of character, possibly an empty sequence.

An EString is the most general metadata key type, as it places no restrictions to the character sequence.[^Well, there are some minor restrictions that follow from the [[metadata syntax|00001006010000]].]

=== Allowed values
All printable characters are allowed.

=== Query comparison
=== Query operator
All comparisons are done case-insensitive, i.e. ""hell"" will be the prefix of ""Hello"".

=== Sorting
To sort two values, the underlying encoding is used to determine which value is less than the other.

Uppercase letters are typically interpreted as less than their corresponding lowercase letters, i.e. ``A < a``.

Comparison is done character-wise by finding the first difference in the respective character sequence.
For example, ``abc > aBc``.

id: 00001006032000
title: Identifier Key Type
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
created: 20210212135017
modified: 20230419175535
modified: 20220914134914

Values of this type denote a [[zettel identifier|00001006050000]].

=== Allowed values
Must be a sequence of 14 digits (""0""--""9"").

=== Query comparison
=== Query operator
Comparison is done with the string representation of the identifiers.
For example, ""000010"" matches ""[[00001006032000]]"".

=== Sorting
Sorting is done by comparing the [[String|00001006033500]] values.

If both values are identifiers, this works well because both have the same length.

id: 00001006032500
title: IdentifierSet Key Type
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
created: 20210212135017
modified: 20230419175551
modified: 20220914131354

Values of this type denote a (sorted) set of [[zettel identifier|00001006050000]].

A set is different to a list, as no duplicate values are allowed.

=== Allowed values
Must be at least one sequence of 14 digits (""0""--""9""), separated by space characters.

=== Query comparison
=== Query operator
A value matches an identifier set value, if the value matches any of the identifier set values.

For example, ""000010060325"" is a prefix ""[[00001006032000]] [[00001006032500]]"".

=== Sorting
Sorting is done by comparing the [[String|00001006033500]] values.

id: 00001006033000
title: Number Key Type
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
created: 20210212135017
modified: 20230419175623
modified: 20220914131211

Values of this type denote a numeric integer value.

=== Allowed values
Must be a sequence of digits (""0""--""9""), optionally prefixed with a ""-"" or a ""+"" character.

=== Query comparison
All comparisons are done on the given string representation of the number, ""+12"" will be treated as a different number ""12"".
=== Query operator
All comparisons are done on the given string representation of the number, ""+12"" will be treated as a different number of ""12"".

=== Sorting
Sorting is done by comparing the numeric values.

id: 00001006033500
title: String Key Type
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
created: 20210212135017
modified: 20230419175633
modified: 20220914130505

Values of this type are just a sequence of character, but not an empty sequence.

=== Allowed values
All printable characters are allowed.
There must be at least one such character.

=== Query comparison
=== Query operator
All comparisons are done case-insensitive, i.e. ""hell"" will be the prefix of ""Hello"".

=== Sorting
To sort two values, the underlying encoding is used to determine which value is less than the other.

Uppercase letters are typically interpreted as less than their corresponding lowercase letters, i.e. ``A < a``.

Comparison is done character-wise by finding the first difference in the respective character sequence.
For example, ``abc > aBc``.

id: 00001006034000
title: TagSet Key Type
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
created: 20210212135017
modified: 20230419175642
modified: 20220914131048

Values of this type denote a (sorted) set of tags.

A set is different to a list, as no duplicate values are allowed.

=== Allowed values
Every tag must must begin with the number sign character (""''#''"", U+0023), followed by at least one printable character.
Tags are separated by space characters.

All characters are mapped to their lower case values.

=== Query comparison
=== Query operator
All comparisons are done case-sensitive, i.e. ""#hell"" will not be the prefix of ""#Hello"".

=== Sorting
Sorting is done by comparing the [[String|00001006033500]] values.

id: 00001006034500
title: Timestamp Key Type
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
created: 20210212135017
modified: 20230419175713
modified: 20220914130919

Values of this type denote a point in time.

=== Allowed values
Must be a sequence of 14 digits (""0""--""9"") (same as an [[Identifier|00001006032000]]), with the restriction that is conforms to the pattern ""YYYYMMDDhhmmss"".

* YYYY is the year,
* MM is the month,
* DD is the day,
* hh is the hour,
* mm is the minute,
* ss is the second.

=== Query comparison
=== Query operator
All comparisons assume that up to 14 digits are given.
Comparison is done through the string representation.

=== Sorting
Sorting is done by comparing the [[String|00001006033500]] values.

If both values are timestamp values, this works well because both have the same length.

id: 00001006035000
title: URL Key Type
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
created: 20210212135017
modified: 20230419175725
modified: 20220914130809

Values of this type denote an URL.

=== Allowed values
All characters of an URL / URI are allowed.

=== Query comparison
=== Query operator
All comparisons are done case-insensitive.
For example, ""hello"" is the suffix of ""http://example.com/Hello"".

=== Sorting
Sorting is done by comparing the [[String|00001006033500]] values.

id: 00001006035500
title: Word Key Type
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
created: 20210212135017
modified: 20230419175735
modified: 20220914130655

Values of this type denote a single word.

=== Allowed values
Must be a non-empty sequence of characters, but without the space character.

All characters are mapped to their lower case values.

=== Query comparison
=== Query operator
All comparisons are done case-insensitive, i.e. ""hell"" will be the prefix of ""Hello"".

=== Sorting
Sorting is done by comparing the [[String|00001006033500]] values.

id: 00001006036000
title: WordSet Key Type
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
created: 20210212135017
modified: 20230419175745
modified: 20220914130725

Values of this type denote a (sorted) set of [[words|00001006035500]].

A set is different to a list, as no duplicate values are allowed.

=== Allowed values
Must be a sequence of at least one word, separated by space characters.

=== Query comparison
=== Query operator
All comparisons are done case-insensitive, i.e. ""hell"" will be the prefix of ""World, Hello"".

=== Sorting
Sorting is done by comparing the [[String|00001006033500]] values.

id: 00001006036500
title: Zettelmarkup Key Type
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
created: 20210212135017
modified: 20230419175441
modified: 20220914135405

Values of this type are [[String|00001006033500]] values, interpreted as [[Zettelmarkup|00001007000000]].

=== Allowed values
All printable characters are allowed.
There must be at least one such character.

=== Query comparison
=== Query operator
Comparison is done similar to the full-text search: both the value to compare and the metadata value are normalized according to Unicode NKFD, ignoring everything except letters and numbers.
Letters are mapped to the corresponding lower-case value.

For example, ""Brücke"" will be the prefix of ""(Bruckenpfeiler,"".

=== Sorting
To sort two values, the underlying encoding is used to determine which value is less than the other.

Uppercase letters are typically interpreted as less than their corresponding lowercase letters, i.e. ``A < a``.

Comparison is done character-wise by finding the first difference in the respective character sequence.
For example, ``abc > aBc``.

They begin with at least three tilde characters (""''~''"", U+007E) at the first position of a line.

You can add some [[attributes|00001007050000]] on the beginning line of a verbatim block, following the initiating characters.
The evaluation block supports the default attribute[^Depending on the syntax value.]: when given, all spaces in the text are rendered in HTML as open box characters (U+2423).
If you want to give only one attribute and this attribute is the generic attribute, you can omit the most of the attribute syntax and just specify the value.
It will be interpreted as a [[syntax|00001008000000]] value to evaluate its content.
Not all syntax values are supported by Zettelstore.[^Currently just ""[[draw|00001008050000]]"".]
The main reason for an evaluation block is to be used with external software via the [[sz encoding|00001012920516]].
The main reason for an evaluation block is to be used with external software via the [[Sexpr encoding|00001012920516]].

Any other character in this line will be ignored

Text following the beginning line will not be interpreted, until a line begins with at least the same number of the same characters given at the beginning line.
This allows to enter some tilde characters in the text that should not be interpreted.

For example:

You can add some [[attributes|00001007050000]] on the beginning line of a verbatim block, following the initiating characters.
A math-mode block supports the default attribute[^Depending on the syntax value.]: when given, all spaces in the text are rendered in HTML as open box characters (U+2423).
If you want to give only one attribute and this attribute is the generic attribute, you can omit the most of the attribute syntax and just specify the value.
It will be interpreted as a [[syntax|00001008000000]] value to evaluate its content.
Alternatively, you could provide an attribute with the key ""syntax"" and use the value to specify the syntax.
Not all syntax values are supported by Zettelstore.[^Currently: none.]
External software might support several values via the [[sz encoding|00001012920516]].
External software might support several values via the [[Sexpr encoding|00001012920516]].

Any other character in this line will be ignored

Text following the beginning line will not be interpreted, until a line begins with at least the same number of the same characters given at the beginning line.
This allows to enter some dollar-sign characters in the text that should not be interpreted.

For example:

id: 00001007700000
title: Query expression
role: manual
tags: #manual #search #zettelstore
syntax: zmk
created: 20220805150154
modified: 20230420190257
modified: 20220913135434

A query expression allows you to search for specific zettel and to perform some actions on them.
You may select zettel based on a full-text search, based on specific metadata values, or both.

A query expression consists of an optional __context expression__, a __search expression__ and an optional __action list__.
The latter two are separated by a vertical bar character (""''|''"", U+007C).
A query expression consists of a __search expression__ and of an optional __action list__.
Both are separated by a vertical bar character (""''|''"", U+007C).

A query expression follows a [[formal syntax|00001007780000]].

=== Context expression

An context expression starts with the keyword ''CONTEXT'', one or more space characters, and a [[zettel identifier|00001006050000]].

Optionally you may specify some context details, separated by space character. These are:
* ''BACKWARD'': search for context only though backward links,
* ''FORWARD'': search for context only through forward links,
* ''COST'', one or more space characters, and a positive integer: set the maximum __cost__ (default: 17),
* ''MAX'', one or more space characters, and a positive integer: set the maximum number of context zettel (default: 200).

If no ''BACKWARD'' and ''FORWARD'' is specified, a search for context zettel will be done though backward and forward links.

The cost of a context zettel is calculated iteratively:
* The specified zettel hast a cost of one.
* A zettel found as a single folge zettel or single precursor zettel has the cost of the originating zettel, plus one.
* A zettel found as a single successor zettel or single predecessor zettel has the cost of the originating zettel, plus two.
* A zettel found via another link without being part of a [[set of zettel identifier|00001006032500]], has the cost of the originating zettel, plus three.
* A zettel which is part of a set of zettel identifier, has the cost of the originating zettel, plus one of the three choices above and multiplied with roughly a logarithmic value based on the size of the set.
* A zettel with the same tag, has the cost of the originating zettel, plus the number of zettel with the same tag (if it is less than eight), or the cost of the originating zettel plus two, multiplied by number of zettel with the same tag divided by four.

Despite its possibly complicated structure, this algorithm ensures in practice that the zettel context is a list of zettel, where the first elements are ""near"" to the specified zettel and the last elements are more ""distant"" to the specified zettel.
It also penalties a zettel that acts as a ""hub"" to other zettel, to make it more likely that only relevant zettel appear on the context list.

=== Search expression

In its simplest form, a search expression just contains a string to be search for with the help of a full-text search.
For example, the string ''syntax'' will search for all zettel containing the word ""syntax"".

If you want to search for all zettel with a title containing the word ""syntax"", you must specify ''title:syntax''.
""title"" names the [[metadata key|00001006010000]], in this case the [[supported metadata key ""title""|00001006020000#title]].
The colon character (""'':''"") is a [[search operator|00001007705000]], in this example to specify a match.
""syntax"" is the [[search value|00001007706000]] that must match to the value of the given metadata key, here ""title"".

A search expression may contain more than one search term, such as ''title:syntax''.
Search terms must be separated by one or more space characters, for example ''title:syntax title:search''.
All terms of a select expression must be true so that a zettel is selected.

* [[Search terms|00001007702000]]
* [[Search operator|00001007705000]]
* [[Search value|00001007706000]]

Here are [[some examples|00001007790000]] of search / context expressions, which can be used to manage a Zettelstore:
Here are [[some examples|00001007790000]] of search expressions, which can be used to manage a Zettelstore:
{{{00001007790000}}}

=== Action List

With a search expression, a list of zettel is selected.
Actions allow to modify this list to a certain degree.

id: 00001007702000
title: Search term
role: manual
tags: #manual #search #zettelstore
syntax: zmk
created: 20220805150154
modified: 20230420153154
modified: 20220821163727

A search term allows you to specify one search restriction.
The result [[search expression|00001007700000]], which contains more than one search term, will be the applications of all restrictions.

A search term can be one of the following (the first three term are collectively called __search literals__):
* A metadata-based search, by specifying the name of a [[metadata key|00001006010000]], followed by a [[search operator|00001007705000]], followed by an optional [[search value|00001007706000]].

  Is true, if zettel metadata contains / does not contain the given key.
* The string ''OR'' signals that following search literals may occur alternatively in the result.

  Since search literals may be negated, it is possible to form any boolean search expression.
  Any search expression will be in a [[disjunctive normal form|https://en.wikipedia.org/wiki/Disjunctive_normal_form]].

  It has no effect on the following search terms initiated with a special uppercase word.
* The string ''PICK'', followed by a non-empty sequence of spaces and a number greater zero (called ""N"").

  This will pick randomly N elements of the result list, preserving the order of that list.
  A zero value of N will produce the same result as if nothing was specified.
  If specified multiple times, the lower value takes precedence.

  Example: ''PICK 5 PICK 3'' will be interpreted as ''PICK 3''.
* The string ''ORDER'', followed by a non-empty sequence of spaces and the name of a metadata key, will specify an ordering of the result list.
  If you include the string ''REVERSE'' after ''ORDER'' but before the metadata key, the ordering will be reversed.

  Example: ''ORDER published'' will order the resulting list based on the publishing data, while ''ORDER REVERSED published'' will return a reversed result order.

  An explicit order field will take precedence over the random order described below.

You may have noted that the specifications of first two items overlap somehow.
This is resolved by the following rule:
* A search term containing no [[search operator character|00001007705000]] is treated as a full-text search.
* The first search operator character found in a search term divides the term into two pieces.
  If the first piece, from the beginning of the search term to the search operator character, is syntactically a metadata key, the search term is treated as a metadata-based search.
* Otherwise, the search term is treated as a full-text search.

If a term like ''PICK'', ''ORDER'', ''ORDER REVERSE'', ''OFFSET'', or ''LIMIT'' is not followed by an appropriate value, it is interpreted as a search value for a full-text search.
If a term like ''ORDER'', ''ORDER REVERSE'', ''OFFSET'', or ''LIMIT'' is not followed by an appropriate value, it is interpreted as a search value for a full-text search. For example, ''ORDER 123'' will search for a zettel conatining the strings ""ORDER"" (case-insensitive) and ""123"".
For example, ''ORDER 123'' will search for a zettel containing the strings ""ORDER"" (case-insensitive) and ""123"".

id: 00001007705000
title: Search operator
role: manual
tags: #manual #search #zettelstore
syntax: zmk
created: 20220805150154
modified: 20230419163812
modified: 20220819194709

A search operator specifies how the comparison of a search value and a zettel should be executed.
Every comparison is done case-insensitive, treating all uppercase letters the same as lowercase letters.

The following are allowed search operator characters:
* The exclamation mark character (""''!''"", U+0021) negates the meaning
* The exclamation mark character (""!"", U+0021) negates the meaning
* The equal sign character (""'' ''"", U+003D) compares on equal content (""equals operator"")
* The tilde character (""''~''"", U+007E) compares on matching (""match operator"")
* The greater-than sign character (""''>''"", U+003E) matches if there is some prefix (""prefix operator"")
* The less-than sign character (""''<''"", U+003C) compares a suffix relationship (""suffix operator"")
* The colon character (""'':''"", U+003A) compares depending on the on the actual [[key type|00001006030000]] (""has operator"").
* The colon character (""'':''"", U+003A) compares on equal words (""has operator"")
  In most cases, it acts as a equals operator, but for some type it acts as the match operator.
* The question mark (""''?''"", U+003F) checks for an existing metadata key (""exist operator"")

Since the exclamation mark character can be combined with the other, there are 14 possible combinations:
Since the exclamation mark character can be combined with the other, there are 10 possible combinations:
# ""''!''"": is an abbreviation of the ""''!~''"" operator.
# ""''~''"": is successful if the search value matched the value to be compared.
# ""''!~''"": is successful if the search value does not match the value to be compared.
# ""''=''"": is successful if the search value is equal to one word of the value to be compared.
# ""''!=''"": is successful if the search value is not equal to any word of the value to be compared.
# ""'':''"": is successful if the search value is equal to one word of the value to be compared.
# ""''!:''"": is successful if the search value is not equal to any word of the value to be compared.
# ""''>''"": is successful if the search value is a prefix of the value to be compared.
# ""''!>''"": is successful if the search value is not a prefix of the value to be compared.
# ""''<''"": is successful if the search value is a suffix of the value to be compared.
# ""''!<''"": is successful if the search value is not a suffix of the value to be compared.
# ""'':''"": is successful if the search value is has/match one word of the value to be compared.
# ""''!:''"": is successful if the search value is not match/has to any word of the value to be compared.
# ""''?''"": is successful if the metadata contains the given key.
# ""''!?''"": is successful if the metadata does not contain the given key.
# ""''''"": a missing search operator can only occur for a full-text search.
  It is equal to the ""''~''"" operator.

id: 00001007780000
title: Formal syntax of query expressions
role: manual
tags: #manual #reference #search #zettelstore
syntax: zmk
created: 20220810144539
modified: 20230411162000
modified: 20220913134024

```
QueryExpression   := ContextExpression? SearchExpression ActionExpression?
QueryExpression  := SearchExpression ActionExpression?
ContextExpression := "CONTEXT" SPACE+ ZID (SPACE+ ContextDetail)*.
ContextDetail     := "BACKWARD"
                   | "FORWARD"
                   | "COST" SPACE+ PosInt
                   | "MAX" SPACE+ PosInt.
SearchExpression  := SearchTerm (SPACE+ SearchTerm)*.
SearchTerm        := SearchOperator? SearchValue
                   | SearchKey SearchOperator SearchValue?
                   | SearchKey ExistOperator
                   | "OR"
                   | "RANDOM"
SearchExpression := SearchTerm (SPACE+ SearchTerm)*.
SearchTerm       := SearchOperator? SearchValue
                  | SearchKey SearchOperator SearchValue?
                  | SearchKey ExistOperator
                  | "OR"
                  | "RANDOM"
                   | "PICK" SPACE+ PosInt
                   | "ORDER" SPACE+ ("REVERSE" SPACE+)? SearchKey
                   | "OFFSET" SPACE+ PosInt
                   | "LIMIT" SPACE+ PosInt.
SearchValue       := Word.
SearchKey         := MetadataKey.
SearchOperator    := '!'
                   | ('!')? ('~' | ':' | '<' | '>').
ExistOperator     := '?'
                   | '!' '?'.
PosInt            := '0'
                   | ('1' .. '9') DIGIT*.
ActionExpression  := '|' (Word (SPACE+ Word)*)?
Word              := NO-SPACE NO-SPACE*
                  | "ORDER" SPACE+ ("REVERSE" SPACE+)? SearchKey
                  | "OFFSET" SPACE+ PosInt
                  | "LIMIT" SPACE+ PosInt.
SearchValue      := Word.
SearchKey        := MetadataKey.
SearchOperator   := '!'
                  | ('!')? ('~' | ':' | '<' | '>').
ExistOperator    := '?'
                  | '!' '?'.
PosInt           := '0'
                  | ('1' .. '9') DIGIT*.
ActionExpression := '|' (Word (SPACE+ Word)*)?
Word             := NO-SPACE NO-SPACE*
```

id: 00001007790000
title: Useful query expressions
role: manual
tags: #example #manual #search #zettelstore
syntax: zmk
created: 20220810144539
modified: 20230420153334
modified: 20220917174956

|= Query Expression |= Meaning
| [[query:role:configuration]] | Zettel that contains some configuration data for the Zettelstore
| [[query:ORDER REVERSE created LIMIT 40]] | 40 recently created zettel
| [[query:ORDER REVERSE published LIMIT 40]] | 40 recently updated zettel
| [[query:PICK 40]] | 40 random zettel, ordered by zettel identifier
| [[query:RANDOM LIMIT 40]] | 40 random zettel
| [[query:dead?]] | Zettel with invalid / dead links
| [[query:backward!? precursor!?]] | Zettel that are not referenced by other zettel
| [[query:tags!?]] | Zettel without tags
| [[query:expire? ORDER expire]] | Zettel with an expire date, ordered from the nearest to the latest
| [[query:CONTEXT 00001007700000]] | Zettel within the context of the [[given zettel|00001007700000]]

id: 00001008050000
title: The "draw" language
title: The ""draw"" language
role: manual
tags: #graphic #manual #zettelstore
syntax: zmk
created: 20220131142036
modified: 20230403123738
modified: 20220311120439

Sometimes, ""a picture is worth a thousand words"".
To create some graphical representations, Zettelmarkup provides a simple mechanism.
Characters like ""''|''"" or ""''-''"" already provide some visual feedback.
For example, to create a picture containing two boxes that are connected via an arrow, the following representation is possible:
```
~~~draw

id: 00001012000000
title: API
role: manual
tags: #api #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20230411164103
modified: 20221219150243

The API (short for ""**A**pplication **P**rogramming **I**nterface"") is the primary way to communicate with a running Zettelstore.
Most integration with other systems and services is done through the API.
The [[web user interface|00001014000000]] is just an alternative, secondary way of interacting with a Zettelstore.

=== Background
The API is HTTP-based and uses plain text and JSON as its main encoding format for exchanging messages between a Zettelstore and its client software.

=== Working with zettel
* [[Create a new zettel|00001012053200]]
* [[Retrieve metadata and content of an existing zettel|00001012053300]]
* [[Retrieve metadata of an existing zettel|00001012053400]]
* [[Retrieve evaluated metadata and content of an existing zettel in various encodings|00001012053500]]
* [[Retrieve parsed metadata and content of an existing zettel in various encodings|00001012053600]]
* [[Retrieve context of an existing zettel|00001012053800]]
* [[Retrieve unlinked references to an existing zettel|00001012053900]]
* [[Retrieve zettel order within an existing zettel|00001012054000]]
* [[Update metadata and content of a zettel|00001012054200]]
* [[Rename a zettel|00001012054400]]
* [[Delete a zettel|00001012054600]]

=== Various helper methods
* [[Retrieve administrative data|00001012070500]]
* [[Execute some commands|00001012080100]]
** [[Check for authentication|00001012080200]]
** [[Refresh internal data|00001012080500]]

id: 00001012050200
title: API: Authenticate a client
role: manual
tags: #api #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20230412150544
modified: 20220107215844

Authentication for future API calls is done by sending a [[user identification|00001010040200]] and a password to the Zettelstore to obtain an [[access token|00001010040700]].
This token has to be used for other API calls.
It is valid for a relatively short amount of time, as configured with the key ''token-lifetime-api'' of the [[startup configuration|00001004010000#token-lifetime-api]] (typically 10 minutes).

The simplest way is to send user identification (''IDENT'') and password (''PASSWORD'') via [[HTTP Basic Authentication|https://tools.ietf.org/html/rfc7617]] and send them to the [[endpoint|00001012920000]] ''/a'' with a POST request:
```sh
# curl -X POST -u IDENT:PASSWORD http://127.0.0.1:23123/a
("Bearer" "eyJhbGciOiJIUzUxMiJ9.eyJfdGsiOjEsImV4cCI6MTY4MTMwNDA2MiwiaWF0IjoxNjgxMzA0MDAyLCJzdWIiOiJvd25lciIsInppZCI6IjIwMjEwNjI5MTYzMzAwIn0.kdF8PdiL50gIPkRD3ovgR6nUXR0-80EKAXcY2zVYgYvryF09iXnNR3zrvYnGzdrArMcnvAYqVvuXtqhQj2jG9g" 600)
{"access_token":"eyJhbGciOiJIUzUxMiJ9.eyJfdGsiOjEsImV4cCI6MTYwMTczMTI3NSwiaWF0IjoxNjAxNzMwNjc1LCJzdWIiOiJhYmMiLCJ6aWQiOiIyMDIwMTAwMzE1MDEwMCJ9.ekhXkvn146P2bMKFQcU-bNlvgbeO6sS39hs6U5EKfjIqnSInkuHYjYAIfUqf_clYRfr6YBlX5izii8XfxV8jhg","token_type":"Bearer","expires_in":600}
```

Some tools, like [[curl|https://curl.haxx.se/]], also allow to specify user identification and password as part of the URL:
```sh
# curl -X POST http://IDENT:PASSWORD@127.0.0.1:23123/a
("Bearer" "eyJhbGciOiJIUzUxMiJ9.eyJfdGsiOjEsImV4cCI6MTY4MTMwNDA4NiwiaWF0IjoxNjgxMzA0MDI2LCJzdWIiOiJvd25lciIsInppZCI6IjIwMjEwNjI5MTYzMzAwIn0.kZd3prYc79dt9efDsrYVHtKrjWyOWvfByjeeUB3hf_vs43V3SNJqmb8k-zTHVNWOK0-5orVPrg2tIAqbXqmkhg" 600)
{"access_token":"eyJhbGciOiJIUzUxMiJ9.eyJfdGsiOjEsImV4cCI6MTYwMTczMTI3NSwiaWF0IjoxNjAxNzMwNjc1LCJzdWIiOiJhYmMiLCJ6aWQiOiIyMDIwMTAwMzE1MDEwMCJ9.ekhXkvn146P2bMKFQcU-bNlvgbeO6sS39hs6U5EKfjIqnSInkuHYjYAIfUqf_clYRfr6YBlX5izii8XfxV8jhg","token_type":"Bearer","expires_in":600}
```

If you do not want to use Basic Authentication, you can also send user identification and password as HTML form data:
```sh
# curl -X POST -d 'username=IDENT&password=PASSWORD' http://127.0.0.1:23123/a
("Bearer" "eyJhbGciOiJIUzUxMiJ9.eyJfdGsiOjEsImV4cCI6MTY4MTMwNDA4OCwiaWF0IjoxNjgxMzA0MDI4LCJzdWIiOiJvd25lciIsInppZCI6IjIwMjEwNjI5MTYzMzAwIn0.qIEyOMFXykCApWtBaqbSESwTL96stWl2LRICiRNAXUjcY-mwx_SSl9L5Fj2FvmrI1K1RBvWehjoq8KZUNjhJ9Q" 600)
{"access_token":"eyJhbGciOiJIUzUxMiJ9.eyJfdGsiOjEsImV4cCI6MTYwMTczMTI3NSwiaWF0IjoxNjAxNzMwNjc1LCJzdWIiOiJhYmMiLCJ6aWQiOiIyMDIwMTAwMzE1MDEwMCJ9.ekhXkvn146P2bMKFQcU-bNlvgbeO6sS39hs6U5EKfjIqnSInkuHYjYAIfUqf_clYRfr6YBlX5izii8XfxV8jhg","token_type":"Bearer","expires_in":600}
```

In all cases, you will receive a list with three elements that will contain all [[relevant data|00001012921000]] to be used for further API calls.
In all cases, you will receive an JSON object will all [[relevant data|00001012921000]] to be used for further API calls.

**Important:** obtaining a token is a time-intensive process.
Zettelstore will delay every request to obtain a token for a certain amount of time.
Please take into account that this request will take approximately 500 milliseconds, under certain circumstances more.

However, if [[authentication is not enabled|00001010040100]] and you send an authentication request, no user identification/password checking is done and you receive an artificial token immediate, without any delay:

```sh
# curl -X POST -u IDENT:PASSWORD http://127.0.0.1:23123/a
("Bearer" "freeaccess" 316224000)
{"token":"freeaccess","token_type":"Bearer","expires_in":316224000}
```

In this case, it is even possible to omit the user identification/password.

=== HTTP Status codes
In all cases of successful authentication, a list is returned, which contains the token as the second element.
In all cases of successful authentication, a JSON object is returned, which contains the token under the key ''"token"''.
A successful authentication is signaled with the HTTP status code 200, as usual.

Other status codes possibly send by the Zettelstore:
; ''400''
: Unable to process the request.
  In most cases the form data was invalid.
; ''401''
: Authentication failed.
  Either the user identification is invalid or you provided the wrong password.
; ''403''
: Authentication is not active.

id: 00001012050400
title: API: Renew an access token
role: manual
tags: #api #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20230412160219
modified: 20220107215751

An access token is only valid for a certain duration.
Since the [[authentication process|00001012050200]] will need some processing time, there is a way to renew the token without providing full authentication data.

Send a HTTP PUT request to the [[endpoint|00001012920000]] ''/a'' and include the current access token in the ''Authorization'' header:

```sh
# curl -X PUT -H 'Authorization: Bearer TOKEN' http://127.0.0.1:23123/a
("Bearer" "eyJhbGciOiJIUzUxMiJ9.eyJfdGsiOjEsImV4cCI6MTY4MTMwNDA4NiwiaWF0IjoxNjgxMzA0MDI2LCJzdWIiOiJvd25lciIsInppZCI6IjIwMjEwNjI5MTYzMzAwIn0.kZd3prYc79dt9efDsrYVHtKrjWyOWvfByjeeUB3hf_vs43V3SNJqmb8k-zTHVNWOK0-5orVPrg2tIAqbXqmkhg" 456)
{"access_token":"eyJhbGciOiJIUzUxMiJ9.eyJfdGsiOjEsImV4cCI6MTYwMTczMTI3NSwiaWF0IjoxNjAxNzMwNjc1LCJzdWIiOiJhYmMiLCJ6aWQiOiIyMDIwMTAwMzE1MDEwMCJ9.ekhXkvn146P2bMKFQcU-bNlvgbeO6sS39hs6U5EKfjIqnSInkuHYjYAIfUqf_clYRfr6YBlX5izii8XfxV8jhg","token_type":"Bearer","expires_in":456}
```
You may receive a new access token, or the current one if it was obtained not a long time ago.
However, the lifetime of the returned [[access token|00001012921000]] is accurate.

If [[authentication is not enabled|00001010040100]] and you send a renew request, no checking is done and you receive an artificial token immediate, without any delay:

```sh
# curl -X PUT -H 'Authorization: Bearer freeaccess' http://127.0.0.1:23123/a
("Bearer" "freeaccess" 316224000)
{"token":"freeaccess","token_type":"Bearer","expires_in":316224000}
```

In this case, it is even possible to omit the access token.

=== HTTP Status codes
; ''200''
: Renew process was successful, the body contains a [[list|00001012921000]] with the relevant data.
: Renew process was successful, the body contains an [[appropriate JSON object|00001012921000]].
; ''400''
: The renew process was not successful.
  There are several reasons for this.
  Maybe access bearer token was not valid.

  Probably you should [[authenticate|00001012050200]] again with user identification and password.

id: 00001012051200
title: API: List all zettel
role: manual
tags: #api #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20230420160640
modified: 20221219154400

To list all zettel just send a HTTP GET request to the [[endpoint|00001012920000]] ''/z''[^If [[authentication is enabled|00001010040100]], you must include the a valid [[access token|00001012050200]] in the ''Authorization'' header].
Always use the endpoint ''/z'' to work with a list of zettel.

Without further specifications, a plain text document is returned, with one line per zettel.
Each line contains in the first 14 characters the [[zettel identifier|00001006050000]].
Separated by a space character, the title of the zettel follows:

```sh
# curl http://127.0.0.1:23123/z
...
00001012051200 API: List all zettel
00001012050600 API: Provide an access token
00001012050400 API: Renew an access token
00001012050200 API: Authenticate a client
...
```

The list is **not** sorted, even in the these examples where it appears to be sorted.
If you want to have it ordered, you must specify it with the help of a [[query expression|00001007700000]] / [[search term|00001007702000]].
See [[Query the list of all zettel|00001012051400]] how to do it.

Alternatively, you may retrieve the list of all zettel as a JSON object by specifying the encoding with the query parameter ''enc=json'':

```sh
# curl 'http://127.0.0.1:23123/z?enc=json'
{"query":"","human":"","list":[{"id":"00001012051200","meta":{"back":"00001012000000","backward":"00001012000000 00001012920000","box-number":"1","created":"20210126175322","forward":"00001006020000 00001006050000 00001007700000 00001010040100 00001012050200 00001012051400 00001012920000 00001012921000 00001014000000","modified":"20221219150626","published":"20221219150626","role":"manual","syntax":"zmk","tags":"#api #manual #zettelstore","title":"API: List all zettel"},"rights":62},{"id":"00001012050600","meta":{"back":"00001012000000 00001012080500","backward":"00001012000000 00001012080500","box-number":"1","created":"00010101000000","forward":"00001012050200 00001012921000","modified":"20220218130020","published":"20220218130020","role":"manual","syntax":"zmk","tags":"#api #manual #zettelstore","title":"API: Provide an access token"},"rights":62},{"id":"00001012050400","meta":{"back":"00001010040700 00001012000000","backward":"00001010040700 00001012000000 00001012920000 00001012921000","box-number":"1","created":"00010101000000","forward":"00001010040100 00001012050200 00001012920000 00001012921000","modified":"20220107215751","published":"20220107215751","role":"manual","syntax":"zmk","tags":"#api #manual #zettelstore","title":"API: Renew an access token"},"rights":62},{"id":"00001012050200","meta":{"back":"00001012000000 00001012050400 00001012050600 00001012051200 00001012051400 00001012053300 00001012053400 00001012053500 00001012053600 00001012080200","backward":"00001010040700 00001012000000 00001012050400 00001012050600 00001012051200 00001012051400 00001012053300 00001012053400 00001012053500 00001012053600 00001012080200 00001012920000 00001012921000","box-number":"1","created":"00010101000000","forward":"00001004010000 00001010040100 00001010040200 00001010040700 00001012920000 00001012921000","modified":"20220107215844","published":"20220107215844","role":"manual","syntax":"zmk","tags":"#api #manual #zettelstore","title":"API: Authenticate a client"},"rights":62}, ...]}
```

      },
      "rights": 62
    }
  ]
}
```

Again, the list is typically **not** sorted, for example by the zettel identifier.

The JSON object contains a key ''"list"'' where its value is a list of zettel JSON objects.
These zettel JSON objects themselves contains the keys ''"id"'' (value is a string containing the [[zettel identifier|00001006050000]]), ''"meta"'' (value as a JSON object), and ''"rights"'' (encodes the [[access rights|00001012921200]] for the given zettel).
The value of key ''"meta"'' effectively contains all metadata of the identified zettel, where metadata keys are encoded as JSON object keys and metadata values encoded as JSON strings.

The JSON objects keys ''"query"'' and ''"human"'' will be explained later in this manual.

=== Note

id: 00001012051400
title: API: Query the list of all zettel
role: manual
tags: #api #manual #zettelstore
syntax: zmk
created: 20220912111111
modified: 20230420160857
modified: 20230116183844

The [[endpoint|00001012920000]] ''/z'' also allows you to filter the list of all zettel[^If [[authentication is enabled|00001010040100]], you must include the a valid [[access token|00001012050200]] in the ''Authorization'' header] and optionally to provide some actions.

A [[query|00001007700000]] is an optional [[search expression|00001007700000#search-expression]], together with an optional [[list of actions|00001007700000#action-list]] (described below).
An empty search expression will select all zettel.
An empty list of action, or no valid action, returns the list of all selected zettel metadata.

Search expression and action list are separated by a vertical bar character (""''|''"", U+007C), and must be given with the query parameter ''q''.

The query parameter ""''q''"" allows you to specify [[query expressions|00001007700000]] for a full-text search of all zettel content and/or restricting the search according to specific metadata.

It is allowed to specify this query parameter more than once.

This parameter loosely resembles the search form of the [[web user interface|00001014000000]] or those of [[Zettelmarkup's Query Transclusion|00001007031140]].

For example, if you want to retrieve all zettel that contain the string ""API"" in its title, your request will be:
```sh
# curl 'http://127.0.0.1:23123/z?q=title%3AAPI+ORDER+REVERSE+id+OFFSET+1'
# curl 'http://127.0.0.1:23123/z?q=title%3AAPI'
00001012921000 API: JSON structure of an access token
00001012920500 Formats available by the API
00001012920000 Endpoints used by the API
...
```

If you want to retrieve a JSON document:

```sh
# curl 'http://127.0.0.1:23123/z?q=title%3AAPI+ORDER+REVERSE+id+OFFSET+1&enc=json'
{"query":"title:API ORDER REVERSE id OFFSET 1","human":"title HAS API ORDER REVERSE id OFFSET 1","list":[{"id":"00001012921200","meta":{"back":"00001012051200 00001012051400 00001012053300 00001012053400 00001012053800 00001012053900 00001012054000","backward":"00001012051200 00001012051400 00001012053300 00001012053400 00001012053800 00001012053900 00001012054000","box-number":"1","created":"00010101000000","forward":"00001003000000 00001006020400 00001010000000 00001010040100 00001010040200 00001010070200 00001010070300","modified":"20220201171959","published":"20220201171959","role":"manual","syntax":"zmk","tags":"#api #manual #reference #zettelstore","title":"API: Encoding of Zettel Access Rights"},"rights":62},{"id":"00001012921000","meta":{"back":"00001012050600 00001012051200","backward":"00001012050200 00001012050400 00001012050600 00001012051200","box-number":"1","created":"00010101000000","forward":"00001012050200 00001012050400","published":"00010101000000","role":"manual","syntax":"zmk","tags":"#api #manual #reference #zettelstore","title":"API: JSON structure of an access token"},"rights":62}, ...]
# curl 'http://127.0.0.1:23123/z?q=title%3AAPI&enc=json'
{"query":"title:API","human":"title HAS API","list":[{"id":"00001012921200","meta":{"back":"00001012051200 00001012051400 00001012053300 00001012053400 00001012053800 00001012053900 00001012054000","backward":"00001012051200 00001012051400 00001012053300 00001012053400 00001012053800 00001012053900 00001012054000","box-number":"1","created":"00010101000000","forward":"00001003000000 00001006020400 00001010000000 00001010040100 00001010040200 00001010070200 00001010070300","modified":"20220201171959","published":"20220201171959","role":"manual","syntax":"zmk","tags":"#api #manual #reference #zettelstore","title":"API: Encoding of Zettel Access Rights"},"rights":62},{"id":"00001012921000","meta":{"back":"00001012050600 00001012051200","backward":"00001012050200 00001012050400 00001012050600 00001012051200","box-number":"1","created":"00010101000000","forward":"00001012050200 00001012050400","published":"00010101000000","role":"manual","syntax":"zmk","tags":"#api #manual #reference #zettelstore","title":"API: JSON structure of an access token"},"rights":62}, ...]
```

The JSON object contains a key ''"list"'' where its value is a list of zettel JSON objects.
These zettel JSON objects themselves contains the keys ''"id"'' (value is a string containing the [[zettel identifier|00001006050000]]), ''"meta"'' (value as a JSON object), and ''"rights"'' (encodes the [[access rights|00001012921200]] for the given zettel).
The value of key ''"meta"'' effectively contains all metadata of the identified zettel, where metadata keys are encoded as JSON object keys and metadata values encoded as JSON strings.

Additionally, the JSON object contains the keys ''"query"'' and ''"human"'' with a string value.
Both will contain a textual description of the underlying query if you select only some zettel with a [[query expression|00001007700000]].
Without a selection, the values are the empty string.
''"query"'' returns the normalized query expression itself, while ''"human"'' is the normalized query expression to be read by humans.


An implicit precondition is that the zettel must contain the given metadata key.
For a metadata key like [[''title''|00001006020000#title]], which have a default value, this precondition should always be true.
For a metadata key like [[''title''|00001006020000#title]], which has a default value, this precondition should always be true.
But the situation is different for a key like [[''url''|00001006020000#url]].
Both ``curl 'http://localhost:23123/z?q=url%3A'`` and ``curl 'http://localhost:23123/z?q=url%3A!'`` may result in an empty list.


As an example for a query action, to list all roles used in the Zettelstore, send a HTTP GET request to the endpoint ''/z?q=|role''.

```sh
# curl 'http://127.0.0.1:23123/z?q=|role'
configuration	00001000000100 00000000090002 00000000090000 00000000040001 00000000025001 00000000020001 00000000000100 00000000000092 00000000000090 00000000000006 00000000000005 00000000000004 00000000000001
manual	00001018000000 00001017000000 00001014000000 00001012921200 00001012921000 00001012920800 00001012920588 00001012920584 00001012920582 00001012920522 00001012920519 00001012920516 00001012920513 00001012920510 00001012920503 00001012920500 00001012920000 00001012080500 00001012080200 00001012080100 00001012070500 00001012054600 00001012054400 00001012054200 00001012054000 00001012053900 00001012053800 00001012053600 00001012053500 00001012053400 00001012053300 00001012053200 00001012051400 00001012051200 00001012050600 00001012050400 00001012050200 00001012000000 00001010090100 00001010070600 00001010070400 00001010070300 00001010070200 00001010040700 00001010040400 00001010040200 00001010040100 00001010000000 00001008050000 00001008010500 00001008010000 00001008000000 00001007990000 00001007906000 00001007903000 00001007900000 00001007800000 00001007790000 00001007780000 00001007706000 00001007705000 00001007702000 00001007700000 00001007050200 00001007050100 00001007050000 00001007040350 00001007040340 00001007040330 00001007040324 00001007040322 00001007040320 00001007040310 00001007040300 00001007040200 00001007040100 00001007040000 00001007031400 00001007031300 00001007031200 00001007031140 00001007031110 00001007031100 00001007031000 00001007030900 00001007030800 00001007030700 00001007030600 00001007030500 00001007030400 00001007030300 00001007030200 00001007030100 00001007030000 00001007020000 00001007010000 00001007000000 00001006055000 00001006050000 00001006036500 00001006036000 00001006035500 00001006035000 00001006034500 00001006034000 00001006033500 00001006033000 00001006032500 00001006032000 00001006031500 00001006031000 00001006030500 00001006030000 00001006020400 00001006020100 00001006020000 00001006010000 00001006000000 00001005090000 00001005000000 00001004101000 00001004100000 00001004059900 00001004059700 00001004051400 00001004051200 00001004051100 00001004051000 00001004050400 00001004050200 00001004050000 00001004020200 00001004020000 00001004011600 00001004011400 00001004011200 00001004010000 00001004000000 00001003600000 00001003315000 00001003310000 00001003305000 00001003300000 00001003000000 00001002000000 00001001000000 00001000000000
zettel	00010000000000 00000000090001
```

The result is a text file.
The first word, separated by a horizontal tab (U+0009) contains the role name.
The rest of the line consists of zettel identifier, where the corresponding zettel have this role.
Zettel identifier are separated by a space character (U+0020).

Please note that the list is **not** sorted by the role name, so the same request might result in a different order.
If you want a sorted list, you could sort it on the command line (``curl 'http://127.0.0.1:23123/z?q=|role' | sort``) or within the software that made the call to the Zettelstore.

Of course, this list can also be returned as a JSON object:
Of course, this list can be returned as a JSON object:

```sh
# curl 'http://127.0.0.1:23123/z?q=|role?enc=json'
{"map":{"configuration":["00000000090002","00000000090000", ... ,"00000000000001"],"manual":["00001014000000", ... ,"00001000000000"],"zettel":["00010000000000", ... ,"00001012070500","00000000090001"]}}
```

The JSON object only contains the key ''"map"'' with the value of another object.

id: 00001012053500
title: API: Retrieve evaluated metadata and content of an existing zettel in various encodings
role: manual
tags: #api #manual #zettelstore
syntax: zmk
created: 20210726174524
modified: 20230109105303

The [[endpoint|00001012920000]] to work with evaluated metadata and content of a specific zettel is ''/z/{ID}'', where ''{ID}'' is a placeholder for the [[zettel identifier|00001006050000]].

For example, to retrieve some evaluated data about this zettel you are currently viewing in [[Sz encoding|00001012920516]], just send a HTTP GET request to the endpoint ''/z/00001012053500''[^If [[authentication is enabled|00001010040100]], you must include the a valid [[access token|00001012050200]] in the ''Authorization'' header] with the query parameter ''enc=sz''.
For example, to retrieve some evaluated data about this zettel you are currently viewing in [[Sexpr encoding|00001012920516]], just send a HTTP GET request to the endpoint ''/z/00001012053500''[^If [[authentication is enabled|00001010040100]], you must include the a valid [[access token|00001012050200]] in the ''Authorization'' header] with the query parameter ''enc=sexpr''.
If successful, the output is a JSON object:
```sh
# curl 'http://127.0.0.1:23123/z/00001012053500?enc=sz'
# curl 'http://127.0.0.1:23123/z/00001012053500?enc=sexpr'
((PARA (TEXT "The") (SPACE) (LINK-ZETTEL () "00001012920000" (TEXT "endpoint")) (SPACE) (TEXT "to") (SPACE) (TEXT "work") (SPACE) (TEXT "with") (SPACE) (TEXT "evaluated") (SPACE) (TEXT "metadata") (SPACE) (TEXT "and") (SPACE) (TEXT "content") (SPACE) (TEXT "of") (SPACE) (TEXT "a") (SPACE) (TEXT "specific") (SPACE) (TEXT "zettel") (SPACE) (TEXT "is") (SPACE) (LITERAL-INPUT () "/z/{ID}") (TEXT ",") (SPACE) (TEXT "where") (SPACE) (LITERAL-INPUT () "{ID}") ...
```

To select another encoding, you must provide the query parameter ''enc=ENCODING''.
Others are ""[[html|00001012920510]]"", ""[[text|00001012920519]]"", and some [[more|00001012920500]].
In addition, you may provide a query parameter ''part=PART'' to select the relevant [[part|00001012920800]] of a zettel.
```sh
# curl 'http://127.0.0.1:23123/z/00001012053500?enc=html&part=zettel'
<html>
<head>
<meta charset="utf-8">
<title>API: Retrieve evaluated metadata and content of an existing zettel in various encodings</title>
<meta name="zs-title" content="API: Retrieve evaluated metadata and content of an existing zettel in various encodings">
<meta name="zs-role" content="manual">
<meta name="zs-tags" content="#api #manual #zettelstore">
<meta name="zs-syntax" content="zmk">
<meta name="zs-back" content="00001006000000 00001012000000 00001012053600">
<meta name="zs-backward" content="00001006000000 00001012000000 00001012053600 00001012920000">
<meta name="zs-box-number" content="1">
<meta name="zs-copyright" content="(c) 2020-present by Detlef Stern <ds@zettelstore.de>">
<meta name="zs-copyright" content="(c) 2020-2022 by Detlef Stern <ds@zettelstore.de>">
<meta name="zs-created" content="20210726174524">
<meta name="zs-forward" content="00001006050000 00001010040100 00001012050200 00001012920000 00001012920500 00001012920503 00001012920510 00001012920519 00001012920800">
<meta name="zs-lang" content="en">
<meta name="zs-license" content="EUPL-1.2-or-later">
<meta name="zs-modified" content="20221219161621">
<meta name="zs-published" content="20221219161621">
<meta name="zs-visibility" content="public">

A __parsed__ zettel is basically an [[unevaluated|00001012053500]] zettel: the zettel is read and analyzed, but its content is not __evaluated__.
By using this endpoint, you are able to retrieve the structure of a zettel before it is evaluated.

For example, to retrieve some data about this zettel you are currently viewing, just send a HTTP GET request to the endpoint ''/z/00001012053600''[^If [[authentication is enabled|00001010040100]], you must include the a valid [[access token|00001012050200]] in the ''Authorization'' header] with the query parameter ''parseonly'' (and other appropriate query parameter).
For example:
```sh
# curl 'http://127.0.0.1:23123/z/00001012053600?enc=sz&parseonly'
# curl 'http://127.0.0.1:23123/z/00001012053600?enc=sexpr&parseonly'
((PARA (TEXT "The") (SPACE) (LINK-ZETTEL () "00001012920000" (TEXT "endpoint")) (SPACE) (TEXT "to") (SPACE) (TEXT "work") (SPACE) (TEXT "with") (SPACE) ...
```

Similar to [[retrieving an encoded zettel|00001012053500]], you can specify an [[encoding|00001012920500]] and state which [[part|00001012920800]] of a zettel you are interested in.
The same default values applies to this endpoint.

=== HTTP Status codes

id: 00001012053800
title: API: Retrieve context of an existing zettel
role: manual
tags: #api #manual #zettelstore
syntax: zmk
created: 20210222151131
modified: 20221208165112

The context of an origin zettel consists of those zettel that are somehow connected to the origin zettel.
Direct connections of an origin zettel to other zettel are visible via [[metadata values|00001006020000]], such as ''backward'', ''forward'' or other values with type [[identifier|00001006032000]] or [[set of identifier|00001006032500]].

The context is defined by a __direction__, a __cost__, and a __limit__:
* Direction: connections are directed.
  For example, the metadata value of ''backward'' lists all zettel that link to the current zettel, while ''forward'' list all zettel to which the current zettel links to.
  When you are only interested in one direction, set the parameter ''dir'' either to the value ""backward"" or ""forward"".
  All other values, including a missing value, is interpreted as ""both"".
* Cost: a normal direct connection has cost 5, an indirect connection is the length of the shortest path between two zettel.
  The cost of a folgezettel or a precursor zettel is just 1, the cost of successor zettel and predecessor zettel are 2.
  You should limit the cost by using the parameter ''cost''.
  Its default value is ""17"".
  A value of ""0"" does disable any cost check.
* Limit: to set an upper bound for the returned context, you should use the parameter ''limit''.
  Its default value is ""200"".
  A value of ""0"" disables does not limit the number of elements returned.

The search for the context of a zettel stops at the [[home zettel|00001004020000#home-zettel]].
This zettel is connected to all other zettel.
If it is included, the context would become too big and therefore unusable.

To retrieve the context of an existing zettel, use the [[endpoint|00001012920000]] ''/x/{ID}''[^Mnemonic: conte**X**t].

````
# curl 'http://127.0.0.1:23123/x/00001012053800?limit=3&dir=forward&cost=5'
{"id": "00001012053800","meta": {...},"rights":62,"list":[{"id": "00001012921000","meta": {...},"rights":62},{"id": "00001012920800","meta": {...},"rights":62},{"id": "00010000000000","meta": {...},"rights":62}]}
````
Formatted, this translates into:[^Metadata (key ''meta'') are hidden to make the overall structure easier to read.]
````json
{
  "id": "00001012053800",
  "meta": {...},
  "rights": 62,
  "list": [
    {
      "id": "00001012921000",
      "meta": {...},
      "rights":62
    },
    {
      "id": "00001012920800",
      "meta": {...},
      "rights":62
    },
    {
      "id": "00010000000000",
      "meta": {...},
      "rights":62
    }
  ]
}
````
=== Keys
The following top-level JSON keys are returned:
; ''id''
: The [[zettel identifier|00001006050000]] for which the context was requested.
; ''meta'':
: The metadata of the zettel, encoded as a JSON object.
; ''rights''
: An integer number that describes the [[access rights|00001012921200]] for the given zettel.
; ''list''
: A list of JSON objects with keys ''id'', ''meta'', and ''rights'' that contains the zettel of the context.

=== HTTP Status codes
; ''200''
: Retrieval was successful, the body contains an appropriate JSON object.
; ''400''
: Request was not valid.
; ''403''
: You are not allowed to retrieve data of the given zettel.
; ''404''
: Zettel not found.
  You probably used a zettel identifier that is not used in the Zettelstore.

id: 00001012920000
title: Endpoints used by the API
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20210126175322
modified: 20230407125812
modified: 20221219162335

All API endpoints conform to the pattern ''[PREFIX]LETTER[/ZETTEL-ID]'', where:
; ''PREFIX''
: is the URL prefix (default: ""/""), configured via the ''url-prefix'' [[startup configuration|00001004010000]],
; ''LETTER''
: is a single letter that specifies the resource type,
; ''ZETTEL-ID''
: is an optional 14 digits string that uniquely [[identify a zettel|00001006050000]].

The following letters are currently in use:

|= Letter:| Without zettel identifier | With [[zettel identifier|00001006050000]] | Mnemonic
| ''a'' | POST: [[client authentication|00001012050200]] | | **A**uthenticate
|       | PUT: [[renew access token|00001012050400]] |
| ''o'' |  | GET: [[list zettel order|00001012054000]] | **O**rder
| ''u'' |  | GET [[unlinked references|00001012053900]] | **U**nlinked
| ''x'' | GET: [[retrieve administrative data|00001012070500]] | | E**x**ecute
| ''x'' | GET: [[retrieve administrative data|00001012070500]] | GET: [[list zettel context|00001012053800]] | Conte**x**t
|       | POST: [[execute command|00001012080100]]
| ''z'' | GET: [[list zettel|00001012051200]] | GET: [[retrieve zettel|00001012053300]] | **Z**ettel
|       | POST: [[create new zettel|00001012053200]] | PUT: [[update a zettel|00001012054200]]
|       |  | DELETE: [[delete zettel|00001012054600]]
|       |  | MOVE: [[rename zettel|00001012054400]]

The full URL will contain either the ""http"" oder ""https"" scheme, a host name, and an optional port number.

The API examples will assume the ""http"" schema, the local host ""127.0.0.1"", the default port ""23123"", and the default empty ''PREFIX'' ""/"".
Therefore, all URLs in the API documentation will begin with ""http://127.0.0.1:23123/"".

id: 00001012920500
title: Encodings available via the API
title: Encodings available via the [[API|00001012000000]]
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20210126175322
modified: 20230403123653
modified: 20230109104839

A zettel representation can be encoded in various formats for further processing.
These will be retrieved via the [[API|00001012000000]].

* [[html|00001012920510]]
* [[md|00001012920513]]
* [[shtml|00001012920525]]
* [[sz|00001012920516]]
* [[sexpr|00001012920516]]
* [[text|00001012920519]]
* [[zjson|00001012920503]] (will be deprecated in v0.11)
* [[zmk|00001012920522]]

id: 00001012920503
title: ZJSON Encoding
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20210126175322
modified: 20230109104722

**Note**: ZJSON encoding will be deprecated in v0.11
---
A zettel representation that allows to process the syntactic structure of a zettel.
It is a JSON-based encoding format, but different to the structures returned by using the plain [[endpoint|00001012920000]] ''/z/{ID}''.

For an example, take a look at the ZJSON encoding of this page, which is available via the ""Info"" sub-page of this zettel: 

* [[//z/00001012920503?enc=zjson&part=zettel]],
* [[//z/00001012920503?enc=zjson&part=meta]],
* [[//z/00001012920503?enc=zjson&part=content]].

If transferred via HTTP, the content type will be ''application/json''.

A full zettel encoding results in a JSON object with two keys: ''"meta"'' and ''"content"''.
Both values are the same as if you have requested just the appropriate [[part|00001012920800]].

=== Encoding of metadata
Metadata encoding results in a JSON object, where each metadata key is mapped to the same JSON object name.
The associated value is itself a JSON object with two names.
The first name ``""`` references the [[metadata key type|00001006030000]].
Depending on the key type, the other name denotes the value of the metadata element.
The meaning of these names is [[well defined|00001012920582]], as well as the [[mapping of key types to used object names|00001012920584]].

=== Encoding of zettel content
The content encoding results in a JSON array of objects, where each objects represents a Zettelmarkup element.

Every [!zettelmarkup|Zettelmarkup] element is encoded as a JSON object.
These objects always contain the empty name ''""'' with a string value describing the type of Zettelmarkup element.
Depending on the type, other one letter names denotes the details of the element.
The meaning of these names is [[well defined|00001012920588]].

id: 00001012920516
title: Sz Encoding
title: Sexpr Encoding
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20220422181104
modified: 20230403161458
modified: 20230109104812

A zettel representation that is a [[s-expression|00001012930000]] (also known as symbolic expression).
A zettel representation that is a [[s-expression|https://en.wikipedia.org/wiki/S-expression]] (also known as symbolic expression).

It is an alternative to the [[ZJSON encoding|00001012920503]][^ZJSON will be deprecated in v0.11].
It is (relatively) easy to parse and contain all relevant information of a zettel, metadata and content.
For example, take a look at the Sz encoding of this page, which is available via the ""Info"" sub-page of this zettel: 
Both encodings are (relatively) easy to parse and contain all relevant information of a zettel, metadata and content.

For example, take a look at the Sexpr encoding of this page, which is available via the ""Info"" sub-page of this zettel: 

* [[//z/00001012920516?enc=sz&part=zettel]],
* [[//z/00001012920516?enc=sz&part=meta]],
* [[//z/00001012920516?enc=sz&part=content]].
* [[//z/00001012920516?enc=sexpr&part=zettel]],
* [[//z/00001012920516?enc=sexpr&part=meta]],
* [[//z/00001012920516?enc=sexpr&part=content]].

If transferred via HTTP, the content type will be ''text/plain''.
Some zettel describe the [[Sz encoding|00001012931000]] in a more detailed way.

=== Syntax of s-expressions
There are only two types of elements: atoms and lists.

A list always starts with the left parenthesis (""''(''"", U+0028) and ends with a right parenthesis (""'')''"", U+0029).
A list may contain a possibly empty sequence of elements, i.e. lists and / or atoms.
If transferred via HTTP, the content type will be ''text/plain''.

There are three syntactic forms for an atom: numbers, symbols and strings.

A number is a non-empty sequence of digits (""0"" ... ""9"").
The smallest number is ``0``, there are no negative numbers.

A symbol is a non-empty sequence of printable characters, except left or right parenthesis.
Unicode characters of the following categories contains printable characters in the above sense: letter (L), number (N), punctuation (P), symbol (S).
Symbols are case-insensitive, i.e. ""''ZETTEL''"" and ""''zettel''"" denote the same symbol.

A string starts with a quotation mark (""''"''"", U+0022), contains a possibly empty sequence of Unicode characters, and ends with a quotation mark.
To allow a string to contain a quotations mark, it must be prefixed by one backslash (""''\\''"", U+005C).
To allow a string to contain a backslash, it also must be prefixed by one backslash.
Unicode characters with a code less than U+FF are encoded by by the sequence ""''\\xNM''"", where ''NM'' is the hex encoding of the character.
Unicode characters with a code less than U+FFFF are encoded by by the sequence ""''\\uNMOP''"", where ''NMOP'' is the hex encoding of the character.
Unicode characters with a code less than U+FFFFFF are encoded by by the sequence ""''\\UNMOPQR''"", where ''NMOPQR'' is the hex encoding of the character.
In addition, the sequence ""''\\t''"" encodes a horizontal tab (U+0009), the sequence ""''\\n''"" encodes a line feed (U+000A).

Atoms are separated by Unicode characters of category separator (Z).

id: 00001012920525
title: SHTML Encoding
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20230316181044
modified: 20230403150657

A zettel representation that is a [[s-expression|00001012930000]], syntactically similar to the [[Sz encoding|00001012920516]], but denotes [[HTML|00001012920510]] semantics.
It is derived from a XML encoding in s-expressions, called [[SXML|https://en.wikipedia.org/wiki/SXML]].

It is (relatively) easy to parse and contains everything to transform it into real HTML.
In contrast to HTML, SHTML is easier to parse and to manipulate.
For example, take a look at the SHTML encoding of this page, which is available via the ""Info"" sub-page of this zettel: 

* [[//z/00001012920525?enc=shtml&part=zettel]],
* [[//z/00001012920525?enc=shtml&part=meta]],
* [[//z/00001012920525?enc=shtml&part=content]].

If transferred via HTTP, the content type will be ''text/plain''.

Internally, if a zettel should be transformed into HTML, the zettel is translated into the [[Sz encoding|00001012920516]], which is transformed into this SHTML encoding to produce the [[HTML encoding|00001012920510]].

=== Syntax of SHTML
There are only two types of elements: atoms and lists, similar to the Sz encoding.

A list always starts with the left parenthesis (""''(''"", U+0028) and ends with a right parenthesis (""'')''"", U+0029).
A list may contain a possibly empty sequence of elements, i.e. lists and / or atoms.
Before the last element of a list of at least to elements, a full stop character (""''.''"", U+002E) signal a pair as the last two elements.
This allows a more space economic storage of data.

An HTML tag like ``< a href="link">Text</a>`` is encoded in SHTML with a list, where the first element is a symbol named a the tag.
The second element is an optional encoding of the tag's attributes.
Further elements are either other tag encodings or a string.
The above tag is encoded as ``(a (@ (href . "link")) "Text")``.
Also possible is to encode the attribute without pairs: ``(a (@ (href "link")) "Text")`` (note the missing full stop character).

id: 00001012920582
title: ZJSON Encoding: List of Valid Metadata Value Objects Names
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
modified: 20230109104713

**Note**: ZJSON encoding will be deprecated in v0.11
---
Every Metadata value element is mapped to a JSON object with some well defined names / keys.

|=Name | JSON Value | Meaning
| ''"\"'' | string | The type of the Zettelmarkup element.
| ''"i"'' | array  | A sequence of [[inline-structured|00001007040000]] elements.
| ''"s"'' | string | The first / major string value of an element.
| ''"y"'' | array  | A set of string values.