Jay Bonci

jaybonci@everything2.com

    (I) Abstract:
  1. Why change the message system
  2. Design goals of the new message system
  3. When is this going to come in?
    (II) General Architectural Changes
  1. Changes to message (opcode)
  2. Changes to message aliases
  3. Changes to editor notes
  4. Changes to massacre and blab
  5. Changes to borging / ignoring
  6. Changes to general messaging
    (III) Feature enhancements
  1. Rooms in usergroups
  2. Message folders
  3. Save sent msgs
    (IV) Nodetypes
  1. msgcommand
  2. msgdelivery
    (V-1) Public Interfaces in Message.pm:
  1. popMessageParam
  2. getMessageOp
  3. doMessageOp
  4. translateAliasName
  5. resolveMessageRecipient
  6. translateAliasNumber
  7. hasMessageBlock
  8. canSendMessage
  9. installMessageBlock
  10. removeMessageBlock
  11. notateNode
  12. unnotateNode
  13. moveMessage
  14. dropMessage
  15. sendMessage
  16. sendMessageToNode
  17. deliverMessage
    (V-2) Private Interfaces in Message.pm
    (VI) Anatomy of a sent message
    (VII) Changelog

(I) Abstract

Why change the message system?

The reasons are many. As our site grows, and our communication needs change, it becomes obvious that the message system to chat with other users on the site needs to be overhauled. Many of the features that worked for the site in the last three years have held the community together and really have watched it grow to the point where those features need to be expanded.

It needs to be overhauled because we need deeper integration of many functions. Right now things like borging, msg-alias following and other items are not seamless for the coders. We need to special case in those items in all areas for them to be followed. Code duplication really is something that I would like to avoid as it makes an already large codebase much harder to maintain

In addition, certain portions of the message system are not modular enough to be maintained efficiently. For one good example message, the opcode, is a gigantic switch statement based upon what type of msg command you are going to send. Therefore if I wanted to work on a /beer command, there won't be any message system until I work out that component (the msg opcode will continue to fail). It's bloated and this overhaul is a good way to resolve it.

Design goals of the new message system

The main numero-uno goal of the message system is complete HTTP-form backwards compatibility with the old system. There will be a time when the old system is still in play and we have a different (but identical) interface to the new system for testing. All commands will go through op="message". This is important as we do not want to have to change the way older clients work, and the general way the E2 forms work. We need to incorporate all the features of the old system into the new one, in a seamless manner to simplify code.

There is also a need to make it easily expandable. In the future, adding easier ways to communicate en masse, and add new features to the system should be interesting and easy. The message system should be almost like a command-prompt interface into E2, much in the same way the Quake prompt works. We'll strive towards that goal with plug-in based modules.

When is this going to come in?

Soon. I'd like it to come in, as there are a lot of people (both here, perlmonks, and people with private ecore) crying for a new system, and this would be a good way to pull all of our concerns together. I don't have a specific time. Testing integration into the existing E2 structure is going to be the hardest part.

(II) General Architectural Changes

Changes to message (opcode)

The message opcode will be changed over to be considerably simpler. The functionality for each message type will be moved to it's own nodetype (see msgcommand below). The job of the message opcode is to make sure that guest user can't execute anything, and to parse the message command into it's msgcommand type. The message opcode will split the message as follows: 

      /msg jb Hey there, hows it going
      ^^^^ ^^^^^^^^^^^^^^^^^^
msgcommand Part sent off to the message plug-in
The system will look a lot like the HTMLCODEs that ecore programmers are used to, but will simplify the mass of regular expressions that we see now. It will also default the system to msg the main room if it doesn't match /\/(\W*?)/.

Changes to message aliases

The message alias system will be ported over to a newer, more expandable system. Currently, we use the chatterbox forward setting. Don't worry, you don't have access unless you're an @, however it's nothing more than a mapping of short names to that of long names, so you're largely not missing out. This setup has the scalability woes of any sort of setting glob in that it's great if you need ALL the items in the settings hash, because you need to parse them to pick them up anyways. However, since we need to only map one to one usually, this may be less than ideal. Message alias translation is a tricky process, and involves several concerns:

  • Mapping non-existent names to numbers: We would like there to be a way of getting a user alias without having to create a user to stopgap it.
  • Mapping numbers to other numbers: This will cause us to walk through the message chain quickly. This won't be a problem, all things considered, because I don't ever expect the message aliasing to get more than two levels deep (an alias to c_e that contains bones... an alias to a usergroup that contains an alias to a person). The message resolution mapping works as follows:
    • If the msg-to name maps to (in order) a user or usergroup, or if not, to see if the short-string maps to a node_id of a bin. There will be two tables of translation, names to numbers and numbers to numbers. We want to work as much as possible off of the numbers table.
    • To see if that node_id maps as an alias. Continue to follow the alias trail until you either see a dead end, loop back on yourself, or you see a "no-resolve" flag come up in the alias table.
    • If the final number maps to a user_id, deliver it to the mailbox. If the number maps to a usergroup, map to each object in the group as a possible alias, and continue to push onto a recipient list (avoiding duplicates). If the number maps to a non-user or a non-usergroup, treat as if it were an editor's note. Otherwise, error out.

Changes to editor notes

Editors have a note feature that we use to keep information on homenodes about who is talking to / working with whom. And it gives ailie a chance to be silly. For those two reasons, we're going to have to keep it around. It's currently listed as a quickly filling up setting which could stand to be moved to it's own system. For that reason alone, we're going to move it to it's own table and have the message system run it. It's a good a time as any to overhaul this system. You'll be able to add editor intel by using a /note command or something similar.

Changes to massacre and blab

There will be changes to the way that massacre and blab work, if nothing more than to simplify the underlying code. We need to make sure that if people have a second account that forwards to their first acct (such as tbbk => TheBooBooKitty), we need to make sure that the Klaproth / blab! messages automatically, and seamlessly forward.

Changes to borging / ignoring

Currently the ignore mechanisms are very over-the-top. They aren't ingrained into the message system anywhere useful. We need to check each time we send a message as to whether or not they are ignoring it. Since our current way of sending messages is a simple sqlInsert into the database, we need the messaging functions to check for that ignorance.

The borg function is similar. We currently check $$VARS{borged} to determine whether someone is borged. $VARS are by no means a good storage mechanism. It's quite volatile, and there are many race conditions associated with it (we will be moving on over to param tables like slashcode does in the future, but for now, it stands as it lies). The check is per message function as to whether you can speak. In the message opcode, in the blab code, etc. This needs to be more uniform

There will be a general table for messages blocks. You can put them in manually, if you would like, in the same manner as your block list currently works (check your Decline Private Messages filter in your Preferences). Borging will be morphed into a forced block put upon you against node_id=0. This means that you won't be able to send any message. There will be a parameter on it that will cause it to expire. Expiration of blocking is another thing that we will have to check. We may percolate this functionality down to normal inward-bound blocking, but for now, it's not a timed thing.

Changes to general messaging

Very little will change about general messaging and chatting except that now if a normal user tries to send a super-user command, it simply won't go through. It will not shout itself to the chatterbox. It's a design choice that has to be made. Typo checking will catch any message that starts with "msg" or ".msg". The likely typos "/smg","/mgs", etc will be aliased in the same manner that "/tell" is aliased to "/msg".

(III) Feature enhancements

Note: these are experimental, and may not make it into the final version of the messaging system

Rooms in usergroups

Due to the abstractions available to us, a possibility is to include certain rooms in usergroups for announcements other other purposes. There are several roundabout uses for this, including creating an announcement list that also announces something to a channel (such as an edev channel). It's tough to imagine uses for it now, as rooms are an underused feature, but the broadcast nature of this may make it worth while in the future.

Message Folders

Similar to the ability to create rooms, creation of folders seems like a likely next step for people. It would utilize the in_room portion of the message table, and (if we made sure the cleanRoom.pl did not wipe them out), we could allow higher-level users (with piles of messages), to get a better way of organizing them (other than turning on Message Inbox Threading, which will still be available).

Sent messages folder

A largely requested feature of our current message system is the ability to save old messages in a "sent" folder of sorts, keepable until they are deleted. These will be up in an off-table for sent messages, until they are deleted as normal.

(IV) New Nodetypes

msgcommand

The new msgcommand nodetype is a modular extension to the message system. It will most likely derive from htmlcode (in the 0.8 version at least), and will run similar to that. It will have to use Everything::Message as it starts, but after that, it should work just fine. The msgcommands will be named in the part of the message after the /, so they could be "msg", "tell", "topic", "borg", "beer", etc to correspond with the appropriate command. They will be called with evalCode as they would normally be called through the htmlcode() function. The pre-1.0 system may use evalCodeX depending on whether they get put in chromatic's compil-o-cache. I have yet to test extensions of nodemethod or htmlcode in the newer version of ecore.

The msgcommand plugins will take one parameter, which is the string after the intial command and a space. It's not the most intelligent parsing scheme, but it's a good standard to adhere to. In the rare case when it's less useful (/me, etc), it'll be easy enough to pre-append the string and move on with life. It's up to each msgcommand to parse that truncated string. We'll provide a parameter to remove the first item(s) from the string, since it's a common usage, but for other plugins (such as /topic), we'll want the whole thing, so by no means will it be standard.

msgdelivery

Nodes of type msgdelivery will also descend from htmlcodes, but will be used when the msg delivery to objects is not completely obvious or essential to the code, and needs to be extended. When a node fails to be a user or usergroup, then the next step is to see if a custom handler is in place for the delivery type. Each node will expand to a type. In pre-1.0 this might be handled by nodemethods, but in 0.8, we'll have a standard type for this. You could use standard messaging to log to either a logfile by overriding the method, or by adding to a wiki, for instance. It is up to the site authors as to how this gets extended.

(V-1) Public interfaces in Message.pm

popMessageParam($str)

    Pops off the first item enclosed in spaces from the string
  • Parameters: The string to be popped
  • Returns: An array. In the 0th element, it returns the element. In the 1nth element, it returns the rest of the string
  • Usage: Primarily a helper function to encapsulate the regex that would have to be run again, and again, and again.

getMessageOp($str)

    Returns an array for the msgcommand to be performed, and the remainder of the message
  • Parameters: The string to be parsed
  • Returns: An array [$msgop, $message]
  • Usage: If it doesn't match /(\W*?), use the default message: "["/chatter", $entirestr]". This will call the default message handler for chatting to a room. This calls popMessageParam to do the initial splitting. This will also strip off the intial "/".

doMessageOp($msgop, $message)

    Calls the correct message handler
  • Parameters: The message op to find as a msgcommand, the message to be sent
  • Returns: Returns success (number of messages sent) or undef on failure of message sending. Note that a blocked message is still a success.
  • Usage: This high level function should only really be called by the message opcode

translateAliasName($str)

    Translates special names into their non-special equivalents.
  • Parameters: The string to translate
  • Returns: The translated string. Note: it's possible (and quite likely), to get a translated string
  • Usage: This translates across a special translation table. If there are any special case names, you would take advantage of them here.

resolveMessageRecipient($str)

    This function resolves the hairiness of node name encoding to one of many possible nodenames.
  • Parameters: The string to try to translate
  • Returns: The resolved node. Can return undef if there is nothing to message by that name.
  • Usage: This will be called internally to handle " " => "_". It will also be used to walk down the chain of "user", "usergroup", etc.

translateAliasNumber($FORNODE)

    Takes a node, returns another node across the alias table... Currently this is implemented via chatterbox forward.
  • Parameters: The node to translate
  • Returns: A node reference. On internal failure (such as a dead-end reference), returns -1.
  • Usage: The aliasing by this node follows all nodes and references to nodes. It will end you up in the right place.

hasMessageBlock($NODE, $USER)

    Checks to see whether the $USER can send a message to $NODE
  • Paramsters: The $NODE to check, and the $USER to check for blockage
  • Returns: True or False
  • Usage: If you are blocked against node 0, whether or not you are checking, you are blocked from sending all messages, and thus you cannot actually send to anyone. If a message block is out of date, it removes it and returns truth.

canSendMessage($NODE, $USER)

    Checks to see whether someone ($USER) can send a message to ($NODE)
  • Parameters: The $NODE to check, and the $USER to check for permission
  • Returns: True or False
  • Usage: In 0.8, this will mainly call $DB->isApproved after checking for a message block against that node. In pre-1.0+, this may have more dynamic messaging (such as one-way usergroups, etc).

installMessageBlock($NODE, $USER, %blockoptions)

    Installs a block such that $USER cannot send to $NODE.
  • Parameters: The $NODE to block against and the $USER to block from. %blockoptions can also be:
    • 'blocked_by' => user_id of the block installer.
    • 'blocked_expires' => timevalue of blockage removal
  • Returns: The $USER blocked against. Undef if internal failure.
  • Usage: This function needs to be secure and probably won't be called outside of the message blocking functionality and the /borg code.

removeMessageBlock($NODE, $USER)

    Removes a message block that prevents $USER from messaging $NODE
  • Parameters: The $NODE on which the block is placed for $USER
  • Returns: The $USER of the previous block, or undef on internal failure
  • Notes: This is mainly an internal function and needs to be secures just as the previous one does.

notateNode($NODE, $USER, $msg)

    Adds a notation to the node.
  • Parameters: The $NODE for which the note is intended, the $USER that is sending it, and the $msg that is to be added to the node.
  • Returns: The notation_id or undef failure
  • Notes: This fails if the $USER already has a note on that node (or optionally replaces it). notateNode ignores aliasing.

unnotateNode($NODE, $USER)

    Adds a notation to the node
  • Parameters: The $NODE for which the note is posted, and the $USER that posted the note
  • Returns: Returns undef if the node doesn't exist, or the node_id of the just removed note.
  • Notes: This fails if there is no note for the user to remove, or on an internal error

moveMessage($MSGID, $USER, $FOLDER)

    Moves a message to a room/folder.
  • Parameters: The $MSGID of the node to be moved, the owner of the message as $USER, and the $FOLDER (folder or room) to move it to. -1 is also a valid user.
  • Returns: Returns the $MSGID of the message actually moved. Returns undef if there is no message, no permission, or on internal error
  • Usage: Typically this would be used in a "move message to folder" functionality.

dropMessage($MSGID, $USER)

    Removes a message from play
  • Parameters: The $MSGID of the message to drop, and the $USER that owns the message. As in the other permission functions, -1 is a valid user (super user).
  • Returns: Returns the $MSGID of the message recently dropped. Returns undef if no permission, no message, or internal error.
  • Usage: This is a common item that will be found in the message opcode.

sendMessage($for_str, $USER, $msg, %msgoptions)

    The high-level message delivery mechanism
  • Parameters: The $for_str in pre-translated, unaliased form, $USER (the one sending the message), $msg the actual text of the message, and %msgoptions, a hash containing various message parameters such as ONO. Exact details to be worked out.
  • Returns: The number of messages sent successfully. A blocked message constitutes success.
  • Usage: This is the function that will be used always to deliver to agents. If you know the exact node that you want to send to (and don't want to bother resolving the titles, use sendMessageToNode();

sendMessageToNode($NODE, $USER, $msg, %msgoptions)

    The other flavor of message delivery. Probably the type to be sent from homenodes, etc
  • Parameters: Same as sendMessage, except takes a NODEREF instead of a string for translation
  • Returns: The number of messages sent successfully. A blocked message constitutes success.
  • Usage: This function is called by sendMessage after it has determined a node to send it to. This function takes over and resolves numerical aliases

deliverMessage($FORNODE, $USER, $msg, %msgoptions)

    The agent for low level message delivery.
  • Parameters: The $FORNODE is a reference to the node for which the message is intended, the $USER is the person sending the node, $msg is the message text and %msgoptions are the message options for items such as ONO, etc.
  • Return: Returns the number of messages delivered. This is the last line of defense. No translation or blocking is performed. FOr that functionality, please use sendMessage. This will rarely be called directly except by member functions. This function differentiates between the lower level nodes and handles things such as dupe checking. This call also invokes overridden handlers if such exist.

(V-2) Private interfaces in Message.pm

    To be documented
  • getDelivery($FORNODE);
  • deliverMessage_user($FORNODE, $FROM, $msg, %msgoptions);
  • deliverMessage_node($FORNODE, $FROM, $msg, %msgoptions);
  • deliverMessage_usergroup($USERGROUP, $FROM, $msg, %msgoptions);
  • isOnline();
  • formatMessage();
  • derefAliasName();
  • derefAliasNode();

(VI) Anatomy of a sent message

(VII) Changelog