Subnostr

Random thought after reading program language garbage collection:

FUSE filesystem, on memory mapped file, allocation is creating a file, freeing is deleting. Memory mapped means the data is opportunistically persisted and enables fast recovery of state to a prior condition.

The logical choice for it is a filesystem that is designed for low latency, memory-like storage, `f2fs` or other similar. Ext-3 and ext-4 are also pretty good choices.

It leads me to the thought, which I have had before, which is essentially eliminating the notion of "working memory" in favour of a fast, SSD based memory cache system, so allocating memory ALWAYS means making one of these new files, freeing always means deleting, and accessing the memory is reading, changing it is writing, there is a notion of a cursor for seeking, and hell, somewhere in the mix, we would have TRIM as well.

Reply to this note

Please Login to reply.

Discussion

Loki 2y ago

It also ties in with the work I am doing this very minute.

I have a segmented, multi-layer message protocol that is processed in various ways, and the basic principle is sentinal magic bytes marking sections, that are consumed from the top and padded out to appear the same length when they are forwarded.

Currently it is using defined, static, structs, and chained methods to encode reading or writing.

Previously I had written encoders that work on the principle of slice of interfaces, in this case, slice of codec.Codec, which is a defined interface type for some unit of data and how to read and write it from disk/wire format to in-memory variables that are addressable.

The dilemma is one I am still not satisfied at all with the construction, I kinda feel like maybe I need to try a third way, that uses method pointers , that an interface type of variable follows it. This way the coding is self-describing.

I just am not sure which way to go with it. My intuition is that the Go implementation of UTF-8 variable length encoding is a model that will make it easier, I want it to be as simple as ReaderWriter types... aargh... well, anyhow, gonna start with reading some of the current Go codebase anyhow. There is several good prototypes for similar stream/object processing code in there.