Home > Technical & Creative Skills > Programming, Servers & Scripts

10+ years of experience in text analysis & scraping - brainstorming ideas (5)

04-20-2017 09:39 AM #1 johncarlof (Member)
10+ years of experience in text analysis & scraping - brainstorming ideas

Hello guys,

the title of this post is not to brag but to stimulate a brainstorming with you and check if these skills can be applied to build something that eventually can be profitable.

I'm a software engineer with many years of experience in developing applications that deal with huge amount of text and perform analysis on them, extract patterns, keywords and other information.

I'm also skilled in scraping data, web browser bot automation and converting different formats and sources.

So a question for you: do you have the need to scrape/cluster/analyze/search in/automatize/whatever you like/ with data sources in text format?

thanks in advance for every feedback I'll get

04-20-2017 11:46 AM #2 caurmen (Administrator)

These sorts of skills can be very useful combined with automation to launch large numbers of extremely targeted niche campaigns.

I recall many years ago one of our founders did something similar where he laser-targeted an enormous number of ringtone campaigns to specific bands.

That sort of thing is usually too man-hour intensive to be worth it, but if you can use your text analysis skills to automate, that makes things a lot easier.

If you've got Machine Learning skills too, then you can apply them to analysing the stats from any campaign: very powerful, particularly at larger volumes. I suspect there are a number of people here who would be interested in having conversations with you about that!

Hope that's useful!

04-20-2017 05:49 PM #3 johncarlof (Member)

Quote Originally Posted by caurmen View Post
These sorts of skills can be very useful combined with automation to launch large numbers of extremely targeted niche campaigns.

I recall many years ago one of our founders did something similar where he laser-targeted an enormous number of ringtone campaigns to specific bands.

That sort of thing is usually too man-hour intensive to be worth it, but if you can use your text analysis skills to automate, that makes things a lot easier.

If you've got Machine Learning skills too, then you can apply them to analysing the stats from any campaign: very powerful, particularly at larger volumes. I suspect there are a number of people here who would be interested in having conversations with you about that!

Hope that's useful!
Hello Caurmen, thanks for your reply.

Yes, machine learning is one of my things. I'm going to find a way to apply these skills in this field.

04-20-2017 06:08 PM #4 cmdeal (Veteran Member)

Answer to your question:

Of course.

Map out the current process that you or anyone else you know goes through. A decent number of these processes will involve entity extraction. A further decent number processes which will involve crude or naive, gut based classification, clustering, and anomaly detection. If you look at what many of the successful software for marketers have done (e.g. Adplexity, kenshoo) pretty much the playbook they followed has been to replace these human tasks with software.

The only issue that you will face is that many of these skills have become quite commoditised and a lot of even free software like Rapidminer can do a lot of this. So you will have to overcome that by targeting a very specific use case that addresses as large a pain point as possible and tailoring your product with laser like precision.

04-20-2017 08:08 PM #5 johncarlof (Member)

Quote Originally Posted by cmdeal View Post

The only issue that you will face is that many of these skills have become quite commoditised and a lot of even free software like Rapidminer can do a lot of this. So you will have to overcome that by targeting a very specific use case that addresses as large a pain point as possible and tailoring your product with laser like precision.
Hey Cmdeal, thanks for this great insights. Not being a commodity should be the n.1 target in careers/work life, you got a great point.

I think it's wise to see these skills as a tool, while money is in the problem's solution implemented with this tool

Let's find the problem :-)

thanks

Home > Technical & Creative Skills > Programming, Servers & Scripts