Monthly Archives: January 2011

Development Update – The State of Release 2

In this post I’d like to give you an update on what is coming in Release 2 and how we’re going to be rolling things out. The work that goes into the development of IMAutomator can be broadly split into two parts. Firstly there is the new feature development so that includes building new tools, adding new features, improving the way the site works and so on. The second half is site support – this means adding sites to the system for the tools to submit to. A submission tool without sites to work with is quite useless.

Work that goes on behind the scenes

Over the last few months a huge amount of time has been taken up with development work that affects our back end code – mainly the submission engine, but also our backup system and various other things. You could say we were the victim of our own success as the number of members seemed to just creep up steadily without being noticed until suddenly everything started to grind to a halt. The submission engine in particular simply could not keep up with the increased work load. Problems like these take up time that you guys don’t see because the work done is behind the scenes.

Since releasing the new engine, we’ve been able to get on track with our Release 2 work. There are two new tools coming – the Article Submitter, which is almost ready for testing, and also a directory submitter but we’re also hitting a bottleneck with site support, particular with the bookmark submitter.

How site support works in IMAutomator

For a site to work with IMAutomator we have to effectively reverse-engineer the submission process so that it can be done automatically by our software and this is not always an easy task. When we first began development we quickly found that many bookmarking sites were based on a template software such as Pligg and Scuttle and for sites based on one of these templates – they nearly all work in the same way or a very similar way. Obviously it’s quicker to add these sites than custom-built ones so we have focused on those. But because it’s very easy to setup one of these sites quickly, people often setup a bookmarking site and then leave it and eventually the domain expires and it is gone. This is the problem we have been facing over the last year – nearly 300 sites added in all and today as I write this post, only 30 remain live in the system, and some of those are PR0 which is not particularly helpful.

We’ve been trying to find ways to outsource the site support work so that more time can be freed in-house to concentrate on the rest of the development but there has been no luck with that so far so it still has to be done in-house. However, the template sites are really starting to prove to be more trouble than they are worth so the longer term plan is to spend the time working on mainstream sites with a much higher PR. A site that has been around for years is hopefully not so likely to disappear a couple of months after we add it.

What we’re aiming to provide

The social bookmarker is our primary tool, so we want that to be the strongest. Our aim is to have at least 50 solid sites in there, mostly PR3-7. Currently the RSS tool needs work – there’s only a handful of sites and some are not working well but we’ve not spent the time on it yet as it not used very much. This is almost certainly due to the fact that many people will only have a single feed to promote but also due to lack of training – if you don’t understand the benefit then you won’t use the tool.

This brings me to another point – training. We’re slowly building out our training page into a more comprehensive resource, but this is a lower priority than development work and site support. The idea is to not only offer comprehensive training on each individual tool, but to also fill in the rest of the details such as – understanding how these backlinks actually help you get traffic, learning to do your SEO properly, and so on. Eventually we want to build our training section into a full-blown resource that will rival many paid products out there.

Going back to the tools, we suspect that after the social bookmarking, the article submitter will be the most popular tool. These two tools can be used in an on-going manner – as long as you have new content, you can bookmark it and build articles with it. The RSS and Directory tools are more one-off tools so will get less use so these will be lower priority on our todo list.

Roll-Out plan for Release 2

The article submitter is ready for testing but we have just a single site in the system right now. We’ll add a few more, and put it into testing in the next couple of days with a handful of sites. While that is being tested, we’ll add some more bookmarking sites. Once these are in testing, we’ll incorporate the directory submitter, again with just a few sites to begin with. After that, we’ll do some much needed work on the RSS submitter and then add as many sites as possible for all of the tools – focusing mainly on bookmark and article sites.

Our target for launch is January 31st. If all we had to do was the development work then that would be no problem but the issue is site support. We don’t want to launch (the price increases with Release 2) if we only have a handful of sites working with each tool as that will not provide enough value for money. On the other hand, Release 2 has been in the works for so long that we really don’t want it to slip into February either.

So the plan is to get the development done and do enough site support to test the new tools and ensure they are working properly and then to concentrate 100% on site support right through to the end of January and into February. As with Release 1, 100 new places will be available and these will be locked in at the Release 2 price. However, the launch does not mean the end of Release 2 – it just means that we bring on more members to use it. From February onwards we’ll be doing more site support work before anything else.

Our aim for the next couple of months is something like this:

  • 50+ Bookmarking sites
  • 40+ Article Sites
  • 20+ RSS Sites
  • 20+ Directory Sites

I hope you’ve all enjoyed this insight into the development process, we’ll keep you posted on the developments over the next few weeks.

Secondary Submissions Bug Fixed

A bug has been found in the secondary submissions code – when data was deleted, such as a site being removed from the system or a member deleting a job, the secondary submissions were removed from the queue but the count of remaining tasks was not being correctly updated. This meant that there were some cases where all of the secondary submissions were complete but the system still thought there was more work to do, causing it to sit effectively in limbo.

We have now fixed the bug and also run a cleanup script to ensure all of the data is correct. Some of you will now find that when you next go into the secondary submissions tool, your latest batch is now complete. You will see the report for the previous batch but you’ll also see the form that allows you to generate a new one.

Tag Extractor Now Works With Hubpages

We have a button on our submission forms that will attempt to pull in the title, description and tags from the page of your URL. In the case of tags it looks for the ‘keywords’ meta tag but there are some sites such as Hubpages that don’t use the keywords meta tags and have their own custom tag format. We’ve now modified our tag extractor to attempt to pull in tags in Hubpages format if the first extraction does not yield results. If there are other formats that do not work with IMAutomator then you can send us the URL and we’ll see what we can do.

Server Outage Last Night

It appears that our server crashed during a midnight (GMT) reboot and the whole site was down for about 10 hours. Everything is up and running now and the engine is working through the backlog and should catchup shortly. Apologies for any inconvenience caused.

Submission Cleanup Run = Jobs Completed

When we have to remove sites from the system, we delete the details of any outstanding submissions to that site from the individual jobs as well as removing the job from the queue. However a bug has been identified that has shown that not all of these were deleted. What this means is that if you have a job containing a submission to a site which has been removed – that job can never complete and it effectively sits in limbo. Not a disastrous bug, but annoying nonetheless.

We have now fixed the deletion bug so all of these tasks should be removed properly but we discovered that there were still some invalid job tasks in the database. We’ve just run a cleanup script to remove these which will automatically have completed jobs that were sitting in the current page waiting for these deleted sites to be submitted. So if you notice that a bunch of your jobs have suddenly completed – this would by why 🙂

Submission Problems Yesterday

Yesterday (4th January) we inadvertently introduced a bug to the submission engine whilst gearing it up for article submissions which caused a bunch of sites to fail! The fix has just been uploaded so the success rates should be back to normal today. Apologies for the inconvenience caused!

Secondary Submission Limits Increased

We have just increased the amount of work that can be submitted in a secondary submissions batch. You can now choose up to 40 jobs and up to a total of 1200 individual submissions.

Stay connected with us in your favorite flavor!