Yearly Archives: 2014

Duplication Error Partial Fix

As well as working on the high server errors that we have been experiencing since we started to add a lot more social bookmarking sites, we have also been working on another issue which is related to this one.

The submission process works in several steps – it logs into the website, submits the job data, in some cases there is a confirmation step and then we check for success and pull out the URL of the created bookmark where available.

The earlier problem of the server timing out and causing the job to fail can actually happen at any point in the submission process. Sometimes the job has been successfully submitted and then the timeout occurs on the final step where we are checking for success. If this happens the job will fail even though the submission did actually succeed. What happens next is that we have an internal retry system that will retry a submission that failed due to a server error several times over a 24-48 hour period. Sites with higher PR are given a higher number of retries.

The problem is that when the job is retried, it is already there (because it was successfully submitted the first time) but now it fails with the error “Duplicate URL”! This is actually an oversight on our part and a bug that has been in the system since the beginning but we didn’t notice it before because the number of failures due to a duplication error was very small.

But since we started adding more sites and started to experience general performance issues, more and more submissions have been failing due to timeouts and this in turn also raised the number of duplication errors. A true duplication error should be very rare, so as these numbers increased it raised a red flag.

So we have now investigated the issue and fixed it – partially. If a duplication error is found then the submission process will now skip ahead to the final step and look for the posted Bookmark URL. However, this will only work if the account that is being used to submit this job is the same account as the one used to submit it earlier when it did indeed succeed because we login to that users account, look at the submission history and pull out the Bookmark URL that was created.

We use a pool of accounts for submission and we rotate around them randomly for all jobs so it may well be a different account that is used on a subsequent retry which means the URL cannot be extracted.

The proper fix for this is to keep track of which account is used for every submission and in the case of a failure, to use the same account when performing any retries. However this is not a trivial fix and it cannot be applied retroactively.

What all of this means is that from now on you will see less failures due to a duplication error (they should now be very rare) but until we can implement the full fix keeping track of the account, the reporting of the created URL will not work in all cases so that field will be blank.

I’ll post again when the next phase of the fix has been implemented and also on the progress of the underlying server timeout errors.

New Social Bookmarking Sites & Performance Issues

Since we removed the Article & RSS submission tools we have been hard at work bringing more social bookmarking sites into the system. Last week we hit 70 sites. However, this seems to have also had an adverse effect on overall system performance.

We monitor the success rates of all sites and monitor the failures carefully and we have noticed that we are starting to see a high number of server errors across the board since we really began to increase the number of sites. At first we thought that the issue was simply that some of the sites were just poor performers in general. This is something we don’t know until we put them into the live system and test them with large volumes of submissions.

So we have been removing those that have very high failure rates but this seems to be just part of the problem. These server errors are almost all time-outs or empty replies from the server which basically means the same thing – we site we are trying to submit to simply is not responding. But what has happened is that the failure rates are increasing across all sites in our system – even those that usually have 100% success rates which means that it appears to be an issue with our own server and not just individual sites.

Right now we’re not sure exactly what the problem is. I don’t think it is related to high volumes as our engine used to process around 8 times more jobs per day than it does now when we had an active free service a couple of years ago. We have the server tech guys looking into it and we are also looking more closely at the log files to try and figure out what is going on.

When a job fails the credits that were spent on that job are refunded so if you have a look at your credit activity log (from the members page) you are likely to see lots of activity there with refunds being given. Also, if a submission to a site fails due to a server error, that site becomes available for submission again. So you will also see the ‘SA’ (Submissions Available) column increasing as the jobs are processed.

You can go into any job that has a SA above 0 and re-select sites for submission to retry them. We also have an internal retry system which will automatically retry a submission a number of times before it fails permanently but this is done over a short period – around 24 hours but you may select these sites for resubmission at any point so please do make use of that feature in order to get more working submissions and in the meantime we shall keep working on the issue and keep you posted!

Article & RSS Submission Tools Now Removed

The last few remaining submissions for RSS feed & Article jobs finished over the weekend and today we have shut down both tools and removed the data associated with them. Thank you very much to everyone who used them over the past few years!

Our focus now is the Social Bookmark submitter which has always been our most popular tool. As of today we’re up to 59 sites and we’re continuing to push that number. It’s actually been quite a while since we even hit 50 bookmark sites so we’re pleased with that. Being able to concentrate on that one tool has helped and we have also been developing some internal tools which has given us access to fresh new sites on a daily basis so we should always have new sites to add as long as they are being built.

Don’t forget that every time we add new sites, those sites are available to all of your existing jobs – you just click the job ID number to edit it and any new sites that have been added since your last submission, along with sites you didn’t select or sites that had previously failed, are now available for submission. Select them just as you would for a new job and re-submit the job to these new sites.

In the jobs list there is a column marked ‘SA’ which is ‘Submissions Available’. This shows how many sites are available for submission for each job. By re-submitting your existing jobs to new sites as they get added to the system you can have a steady stream of fresh links coming in to your content.

The Article & RSS Feed Tools Are Now Read Only

Last week we blogged about the plan to phase out the article and rss feed submission tools. The first phase has been implemented today. The tools are still there but the buttons to submit new jobs has been removed and the resubmission section of the job details screen has been removed.

You may still view the progress of your jobs and edit them, but you cannot submit them to new sites or submit new jobs.

In the meantime we’re continuing to add lots of new bookmark sites. We’re almost at 50 now. Don’t forget to check the ‘SA’ column of your bookmark jobs as this tells you how many new submissions are available. Then just click the job ID, select the sites and resubmit the job to the new sites.

Article & RSS Submission Tools To Be Phased Out Next Month

A little while ago we blogged about allowing our Article & RSS submitter tools to wind down naturally, stating that we would continue to monitor the submissions, removing any sites that stopped working but would not be adding any new sites.

The time has now come where we feel that the current offering does not provide enough benefit to continue. Currently, we have 14 RSS directories and only 10 Article directories. There are a few of you that still use these tools but the amount now is so tiny that it is time to remove them.

Phasing Out In Several Steps

What we will do is remove the tools in several steps:

 Step 1: Disallow new submissions

We will begin simply by removing the functionality to add new jobs to these tools. The actual tools pages will still remain in the system so you can continue to check on the progress of your existing jobs. At this time we will also remove mention of the tools from our sales page, training pages and so on.

We will do this next week on Thursday 27th February 2014 so if you do wish to use those tools, do so within the next few days.

Step 2: Wait for current jobs to finish

We will monitor existing jobs for these tools and will wait until every last submission has been processed before taking any further action. Due to our drip-feed functionality we expect this to take several weeks. Once all the jobs have been processed will will notify members so that you can take once last look at your jobs before we take down the tools entirely.

Step 3: Remove the tools altogether

After all jobs have been completed and members have had enough time to check over them (probably about a week) we will take the tools down entirely. The information about past jobs will no longer be available from our site and we will remove all references to them.

What’s Next?

Going forwards we are going to be focusing heavily on the social bookmarking tool which has always been the most popular. We have recently written some back end tools that allow us to incorporate new sites into the system a little quicker than before so we’re really hoping to increase the number of sites available.

So far this week we have added 8 new sites though sadly we have had to remove some as well. It is always an on-going effort to get the sites list fresh!

Stay connected with us in your favorite flavor!