2018/09 RetroChallenge...

Back several years ago I won the Retro Challenge with my ViewData BBS.
I want to fully re-write it using .Net Core to provide a more streamlined version with the ability to run on other platforms other than Windows. 

My goals for this are as follows...

Cross Platform / Containerisation

I want to be able to run the new system on multiple platforms including cloud based services such as Azure.
I want to be able to possibly containerise the system so that I can easily deploy multiple copies via containers such as Docker.

Pipeline With Modular Framework Components

Create a modular framework that allows pipelining of components together.
Currently the system has a fixed coded pipeline, I want the new system to have no requirement on any component that plugs into the pipeline.
This should allow me to create a generic BBS alongside the ViewData version by switching out components.
I also allows me to insert new functionality into the pipeline just by adding in new components.
The configuration of the pipeline will not be fixed and will be set in the configuration files.

Connectivity

The system needs to be able to handle both direct Telnet and dial up connections.
I intend to create the same sort of gateway for Dial up as before.
I also want to ideally have a web based terminal that can connect to the system. I have seen an example of a Minitel system doing this with Web sockets and may look at creating a gateway to allow this.

Commands

The system will respond to the standard * and # commands that the original Prestel system did. It will also allow for the use of keywords to quickly navigate to pages.
Basic command and navigation will be provided by a module that will allow the user to navigate between pages on the system.

Input Forms

I previously started on a prototype input mechanism that allows for input fields to be defined.
I want to expand on this and provide a framework for creating entry forms and applications.

Graphics

A library of utilities for mapping graphics to areas of the screen would be useful. I would like to be able to load in images / sprites and have the system map them to the correct graphical output. Colour changes and mapping of colours would need to be optimised by the system to get the best output available.

Screen Optimisation

When updating the screen the system creates a buffer on the server side and remembers the state of the screen.
I want to be able to do clever things such as scroll / change areas of the screen without re-drawing the entire screen.
There are commands for moving the client cursor so I want the system to be able to calculate the commands to send to change just the parts of the screen that are necessary to minimise the amount of traffic required and to speed up client rendering. 

Telnet Gateway

The previous system had a telnet gateway that can connect to other online system and pass through the data.
This allows for connecting to other online only ViewData BBSs such as CCL4 and the new Telstar system. 

Telesoftware Implementation

The original Prestel system had a mechanism for downloading software via multiple pages.
A client could run special software that would move through the pages reading various embedded headers and encoded body data to retrieve data.
I would like to have the system be able to index various retro files and allow the client to download them.

Mail System

A mail system that allows users to send (internal) or maybe later full email needs to be implemented. I may try and make it reasonably authentic to the original Prestel by using the built in user ids to send message between users.

Search Engine

An implementation of a simple search engine that allows users to search Google and return results in a paginated way would be nice.

Twitter Feed

I'd like to have various twitter integration points. Ideally the system will tweet events that happen in the system such as users logging in and out and possibly display twitter feeds as pages in the system (these may be built offline and served as static pages).

Background Jobs

To update certain pages I will require a backend agent that updates static pages with content. 
The system may contain template pages that can be used by these agents to create content.
Automatic content such as Weather, news and other content that can be refreshed periodically will be created by background jobs.

Content

The system will need static pages defined to add interesting content to the system.
There are a lot of original pages that I'd like to use as a base for certain parts of the system to give the users an experience as near to the original as possible.

Other 

Lots of other stuff I've either missed or will think of later...

The chance of me actually getting a large amount of this done is not very likely but I am away at the ABUG (Acorn BBC User Group) meeting for the weekend of the 8th so that would give me a reasonable time period to actually start something.

Wakefield Risc OS Show

Got up early as I was staying at the hotel. Wasn't feeling that hot as I'd stayed up drinking with a few of the BBC crowd that were staying at the hotel for the show.

Setup

Brought the boxes with all of my kit to my stand, unfortunately the lift in the hotel wasn't working so we had to climb 3 flights of stairs with everything. Luckily a very nice porter helped with some of the stuff.

Unpacked the Communicator and set it up with the monitor and put the PABX and modem for connecting to under the table hidden out of sight.

Spent the 2 hours before the show opened ensuring that the software was running, connecting my laptop to the hotel WIFI and pointing the dial up gateway to the server backend that sits on a VM in my Manchester data centre.

Tried various times to dial the modem and nothing was happening, the Communicator would go online but it hadn't dialled correctly. My friend Ryan then managed to find an option in the settings for using Tone dialling, we set the option and the thing dialled the modem correctly.

After a few more attempts we got a response from the server side. Unfortunately something was amiss, we were getting formatting missing on the screen.
I think this may be due to the FTDI cable I was using, I had issues with the USB modems that I previously used. I think there is some issue with the actual parity settings it uses VS the ones I've asked it to use. After a bit more fiddling we managed to connect it to CCL4 (a rather random telnet available ViewData BBS).

(Photo from BeebMaster.co.uk)

Interest

Quite a few people we interested in the machine, its a fairly obscure one that not many people had come across. I had a nice conversation with a guy from the Netherlands who is setting up a show in Holland and was looking for people to show items such as this.

Ian (BeebMaster) came over and spent a while figuring out the workings of the machine. It has an interesting set of applications that can be instanciated with a specific amount of memory and unlike the BBC you can switch between applications and they stay resident in memory. 

After taking a load of photos of the machine he ended up taking it apart so that he could get internal photos of the board etc.
We found some interesting things inside, the machine has 2 batteries (which I need to change) one I presume stores the settings not sure what the other is used for (you an store files in the memory so maybe it does that). There is also a pass through connector for a phone on the side but there is a removable panel that is in place so you can't see it.  

2017/04 RetroChallenge...

I'm supposed to be demonstrating a ViewData system at the Wakefield RISC OS show at the end of April where I have a stand booked.

Therefore my Retro Challenge is to sort the stuff I need for the show...

Re-Write BBS Software

I wrote an example BBS system for a previous Retro Challenge (and won :)). I want to re-write the system to make it more expandable and easier to pipeline the components so that they can also be used for generic BBS systems as well as the more specific ViewData one.

Setup Example Pages

Once the system is up and running I need to provide some actual content for people to explore while connected to the system.

ViewData Demonstration Machines

On the last Retro Challenge I wanted to setup a set of 3 demo machines that can be used for connecting to my ViewData BBS. The idea is that I want to have a set of machines setup to show the "Internet" in 1985.
Each setup will consist of a BBC Micro, Cub Monitor and Acorn Prestel adapter.

I have decided that I will concentrate on 1 machine and then do the other 2 if I get time.

Each machine will need...

1) Cleaning
2) RetroBriting
3) Fixing / refurbishing (Probably power caps etc)
4) Configuring

TV

I managed to acquire a nice 80s TV that will sit on the side showing clips and videos from 1980 - 1985.
Originally I wanted to show a mash-up of adverts, cartoons and music videos. After looking into the licencing for this it would require me to find the original copyright holders. Therefore I may look at getting a licence for displaying movies and I think could possibly display the BBCs "Micro Men" and possibly the old Micros Live series using the licence (although I need to verify this).

Phone

A while ago I converted a 746 phone to use tone dialling, I have the parts to do a few more and today just received new cables for them. I would like to therefore setup at least one for the demo.

Phone System

Once everything is setup it needs to be connected together using a PABX. I bought another cheap one and intend to use it exclusively for this task. It will be configured to have several modems connected to the server and the machines connected to the ports to allow dialling into the system. Ideally I want to set it up to ring the next available modem so that I can also connect it to a real external phone line (it has 3 connections so I probably will have 1 real line and 2 voip ones).

Server

I have a small HP Micro server that I want to configure as a portable BBS server, I could however run the system as I do now over a VPN from my server in Manchester. To do this I have a Draytek router that can be configured to use VPN and might be setup as a WIFI client to connect to the wireless on site.

Other Stuff

To make the stand at the show more interesting I need to print out some instructions on how to use the system. Maybe a few movie posters from 1980-1985 and also litter the desk with Prestel guides.

2016/10 RetroChallenge...

I'm going to try fairly simple tasks this time round, the intension is that I actually complete them.

ViewData Demonstration Machines

I want to setup a set of 3 demo machines that can be used for connecting to my ViewData BBS. The idea is that I want to have a set of machines setup to show the "Internet" in 1985.
Each setup will consist of a BBC Micro, Cub Monitor and Acorn Prestel adapter. I also want to get a nice 80s TV that can sit on the side showing clips and videos from 1980 - 1985.

Each machine will need...

1) Cleaning
2) RetroBriting
3) Fixing / refurbishing (Probably power caps etc)
4) Configuring

Kryoflux Imaging

For the last RetroChallenge I setup a PC to do my software imaging using a Kryoflux. I have since sorted the machine and it now has all the software I need to image disks.
I have a large amount of software that needs imaging, I'll start with some of the BBC titles and then continue with the Archimedes software.

Pimped Out Master

I currently have a BBC Master on my desk but I'd like to fully pimp the machine out.
I've also looked at doing a custom black paint job on the machine (I have special Vinyl paints so may give that a go).

So there are my goals, we'll see if I can actually accomplish any of them next month :)

Prestel & ViewData Services

In the UK back before the internet arrived there was a system called Prestel that provided various online facilities via a ViewData interface. This was similar to the Minitel system used in France and was closely related to Teletext.

Here is my Minitel terminal all setup to test my dial in gateway and connect to Level 29 BBS.

And here it is logged into Level 29 BBS. Notice the screen size is very restrictive due to the 40 column width compared to most ASCII terminals that have 80.

Micronet 800

As part of the Prestel service large customers could rent pages (known as frames) or the Prestel mainframe could hand off to an external system to provide other services.

One of the most popular services was Micronet 800, it provided chat rooms, multiplayer online games and message boards about various topics.
It was called Micronet 800 due to that being the entry page number provided by Prestel (you entered 800 to get to the main page within Prestel). 

The main problem with connecting to Micronet / Prestel was that you needed a modem, most home users of the time didn't own a machine with a serial port so in an attempt to get more users Micronet provided a package for the Spectrum. As the Spectrum 48k did not come with a serial port and couldn't display the resolutions required to show the ViewData frames this was a problem. Micronet got round this by providing a modem with built in ROM and hardware to update the Spectrum so that it could use the service.

The modem sits nicely under the Spectrum and connected to the bus at the back.

Modem Functions

When you connect the unit to the Spectrum it patches the ROM to load the Micronet code.
This gives you get a nice splash screen when the machine is turned on.

When pressing a key you are returned to the main system menu.

There is a mode for automatically logging onto the service (this remembers your ID number and enters it at the relevant prompt).
You can save frames, store programs and check the mailbox. The system also allows you to drop back out to Basic so that you can use the Spectrum without having to disconnect the unit.

Selecting log on asks for you ID and then gives you this screen.

At this point you have to manually dial the number to connect the modem.
When you hear the tone you flick the switch on the front of the unit to go online.

Problems Connecting

As the unit is designed to connect to a ViewData system at V23 (1200, 75) it does not have the modern modem facilities such as auto-bauding. It expects the other end to be the same. When trying to connect to Level 29 BBS we hit the problem as the modem on the other end is using Bell protocols and we are using the European CITT ones. The main difference being the tone frequencies that they use.

My modern modem has an AT command to switch modes but will usually just figure out the other ends requirements.

As we don't have that option I altered my ViewData system dial in gateway to connect to Level 29 via telnet. This allows me to dial my machine locally and the software then connects to the BBS via telnet does any character translation required and passes the data back to the Micronet modem.

Connected To Level 29 Via The Gateway

When I used this in my office it worked fine, I could type my user name (with a little bit of effort as the keyboard is a bit awkward on the Spectrum) but my monitor wouldn't keep the sync with the Spectrums video output and the image kept rolling. I then moved this downstairs on a very long telephone extension so that I could use the TV.
I think that something was getting messed up as the keyboard was not sending the correct key presses back to the gateway.

Anyway you can see the main login page that we are connected to. The Minitel terminal in the office works perfectly with this so I think I need to sort a shorter telephone extension cable out and try it again.

Plain Text Gateway

As the dial up part of the system is separate from the main BBS I'm going to look into creating a plain text (ASCII) gateway that removes the ViewData codes and provides a generic interface for standard machines to connect to.

ViewData connects at 1200/75, the system needs to be able to take other speeds including modern modems.

Most of the ViewData characters are compatible with ASCII although there are other mapping that will need to be done to correct certain characters.
There are a whole set of graphics characters that are used to create ViewData graphics, these are basically just standard characters that have been escaped.
ViewData uses escape codes to change colour, change to graphics mode and various other effects such as flashing text. This will need to be removed from the ASCII stream.

Unlike most ASCII BBSs the ViewData screen is a set size and doesn't scroll. Therefore any page will need to ensure that the ASCII screen is cleared before sending any page to the client.

Keyboard entry on the incoming system needs to be mapped to the correct character. Also there are several shortcut keys on most ViewData terminals, these provide navigation such as # *0# etc
The client will need to enter the keyboard commands in full or map a function key to send the correct string to the server.
Basic functionality should work as most pages use the numeric keys and the hash (pound in the US) to move to the next screen.

Once the basic functions are there I may look into getting more feature rich terminals working including maybe getting coloured ASCII working.

Source Control

Because I'm lazy I had a large set of development directories that I date stamped when doing the last RetroChallenge. I decided these should really be under source control so that if I change functionality and decide that code I've changed needs to be reverted back to previous version its much easier as I have a proper list of changes to files over time.

The first problem is that my source control server was destroyed when I moved my server to a new data centre. I have a backup of all the source control files but needed to rebuilt a virtual machine and install the source control software before I could start.

Once done I found all the old versions of my source and sorted them into date order, one by one I copied the files over the top of the previous version and checked them in, ensuring each time to removed unused files that had been deleted or renamed in the project. Each time also ensuring that everything still compiled correctly.
Now I have a nice an clean project with all of the changes I've made over the last RetroChallenge so that any code that has been removed is still there if required.

Once complete I then got a fresh version from source control and ensured everything is working.
At this point I found a few issues that need addressing with hard coded machine names (bad I know) which need to be setup in the configuration file instead.
Everything was working fine apart from the terminal program that I was fiddling with back along, somehow I've managed to mess up the line feed and everything is not displayed where it should be, I think this is the actual terminal program at fault in rendering the frame not the backend. I will need to test this against the other terminal program I have to check that is isn't the back end.

Database

Started on the database for my BBS backend which will hold the system frames and control the functionality within the backend.

Tried to make the frame store as compatible with the current saved frame formats that are out there, most of them contain details of the frame owner, permissions and routes to other frames.

As the ViewData system works on frame numbers even gateways etc will be allocated a frame, therefore I have abstracted out the frame details so that I can base other entities on them for future functionality. Started off by creating a page (based on frame) which can hold the code for a static page. These are date stamped so that the system can tell if the frame has been updated. Backend systems such as news feeds etc can update these pages to change content in the system. Also added a gateway (based on frame) which is a frame that takes the user to another BBS, this holds the remote connection details of the other BBS and will be contacted when you navigate to this specific frame type. This allows me to just add frame addresses that access other systems.

The system also allows for multiple systems to be run from the same database, this allows me to have multiple BBS systems running with different content in each. Ideally this may include examples of old BBSs and example Prestel pages.

Server

Originally the entire BBS ran on a PC in my office, this is mainly due to the numerous modems connected to it.

As I have a dialup gateway as a separate module I'm intending to run the dialup gateway on the PC and then have everything else installed on a new virtual server.
This will enable me to have the backend available even if the dialup lines aren't. The backend and database can then run on the new virtual machine and can then talk to the gateway wherever it is.

So ended up building another virtual machine just for this. Installed SQL, IIS (for the web based interface later on) and later once recompiled, the services that run the BBS.

More ViewData BBS Server

After winning a prize in the last Retro Challenge, I'm intending to continue on with my BBS system and add more functionality and content to it.

Now that the basic ViewData server and BBS system work I intend to do some of the following...

1) Setup a login / signup system and proper welcome page for the BBS.
2) Add some static content and various information pages.
3) Finish the database driven backend so that the pages are served from a database (this will also allow me to run multiple BBS setups with different content).
4) Load pages from archived Prestel data into one of the BBS instances.
5) Create a messaging system (possibly integrate into email).
6) Add a chat area, might have different rooms for conversations.
7) File download area (Possibly using CITT, might also support Xmodem etc).
8) Various feeds providing information such as the news and weather.
9) Gopher client
10) Gateway between BBSs.
11) Some kind of MUD or text adventure.
12) Visitor list and statistics.
13) Message wall for visitors (for WGOODF so that he can show his Dragon was connected :) )

ViewData Terminal Software

I started on a terminal program that provides a ViewData terminal that will connect to the BBS.
It mostly works but needs some polishing. As part of the clean up of the code I want to convert the graphics part so that it works on WPF and Silverlight.
The intention is to then host this on a web page so that you can access the BBS from the web.

Required Items

To start off I need to ensure the following...

1) Ensure that the PC and modems are all setup correctly and ready to go.
2) Re-enable the VOIP lines (I now have several setup so we could expand the BBS to have 4 modems).
3) Solder a set of cables for the BBC to connect to modems. (Had an issue with the connectors I bought, they won't fit in the hole on the modem so need to rethink this.)
4) Try and get terminal server box to accept incoming connections and connect out on modem.

Terminal Hardware

I've already managed to use the following to connect to the BBS...

1) Alcatel TerminalTel 257 VideoTex terminal.
2) Acorn Communicator. 
3) Sinclair Spectrum 48k with Prism VTX 5000 modem.
4) ConnectVX (A windows ViewData client).

Due to cabling issues I didn't manage to get the following connected, the plan is to get the following cabled up and connected...

1) BBC Micro with Acorn Prestel adapter. (Was still waiting for Prestel adapter to arrive during 2014 WW)
2) BBC Micro with Prism modem (1000 & 2000). (Needs cables making, had issues due to size of connector)
3) BBC Micro with Pace Nightingale modem. (Needs rom and cable creating)

I also have the following that I'd like to get connected...

1) Acorn A3000 with old modem.
2) Tatung Einstein + Prism Modem 1000 (Needs comms software)
3) Tandata TD1100 (Needs cleaning up and checking it works)

Acorn Communicator

Finally got my Acorn Communicator setup and dialed the BBS...

As you can see the Communicator is pretty cool, it is a one-per-desk type system designed to be networked using Econet. It has a word processor and spreadsheet built in, can dial numbers automatically and be used as a terminal with various systems.

Entry Forms

Managed to do a little more on the entry forms for the ViewData system.

The system now navigates from field to field and allows for cursor control and correction of entries in each box.
The user can use the cursor keys to edit the entries and move between fields. Keys such as delete also work correctly and the cursor is constrained within the field area when moving around.

Initially I've implemented a text box, today I added a new field type called password which is based on the textbox but hides the characters entered and changed them to set character.

The demo screen below shows the output from the screen where I have entered a bogus user and password and it has been extracted from the field model and displayed at the bottom in white.

This framework is to be the basis of the applications I want to create within the system. The first thing to implement will be a proper login and signup page at the start.