How to create an optimization engine

A place to discuss the implementation and style of computer programs.

Moderators: phlip, Moderators General, Prelates

jcbutler31
Posts: 3
Joined: Sat Aug 02, 2014 4:57 am UTC

How to create an optimization engine

Postby jcbutler31 » Tue Aug 05, 2014 9:02 pm UTC

I've been researching how to make my website for hours a day for the past month. I don't have much, if any, experience in coding besides spending a couple of hours here and there on codecademy. What I am trying to make is a news website wherein a user creates an account and chooses the type of news that caters to his or her interests. After the user identifies his or her interests, the engine generates a feed that displays an article base according to the user's interests. The user would then click on which article(s) he or she wants to read. The site would work much like many other news apps such as ESPN – where you choose the sports and teams you want to receive news for; or Yahoo! Finance – where you choose the stocks or indices you would like to follow. Many other news websites – Facebook users like pages and friend people to receive news from them and Twitter users follow handles to receive news from them – operate like my desired website as well. I understand my website will never be (and doesn't have to be) on the high programming level of those successful sites, but (1) how can I create a website like this without learning a language (e.g. perhaps where to go to hire a professional) and (2) which language would be most beneficial to learn if I didn't have the budget to create a site like this and had to teach myself?

Thanks in advance,
JB

Tub
Posts: 391
Joined: Wed Jul 27, 2011 3:13 pm UTC

Re: How to create an optimization engine

Postby Tub » Wed Aug 06, 2014 2:34 pm UTC

jcbutler31 wrote:(1) how can I create a website like this without learning a language

You could just install wordpress (or similar), tag each article by interest, and use the existing tag filters...

Using existing software would free you from most of the technical burden so you can focus on creating content, attracting and keeping users. Even if the software is not a perfect fit, writing a better one is obviously not your strong point. Focus on your strengths.

jcbutler31 wrote:(e.g. perhaps where to go to hire a professional)

Find someone in your area. If you cannot communicate in person, the results will likely be different from what you expected. You can find them via yellow pages, google, whatever.

Bring a functional specification. A *full* functional specification. This forum post is way too terse. How would the users' interests be modeled? Chosen from a predefined set of possible interests, freeform, ...? What features are you expecting that you haven't yet written down? Like the ability for your users to delete their accounts or reset their passwords? If the feature is not in the spec, it won't be in the product.

And be prepared for the bill, because professionals aren't cheap. Do you have 10.000 bucks?


That being said, don't trust the first company you find. There are companies out there programming worse than a group of monkeys. Ask for references. Research. Did their last projects succeed? Were their clients happy?

jcbutler31 wrote:(2) which language would be most beneficial to learn if I didn't have the budget to create a site like this and had to teach myself?

Depends. How much time do you have?

A master's degree in computer science takes 10 semesters for a reason. Even if you skip the technical backgrounds and focus on a single language for a single purpose, it's not something that can be self-taught in less than a year. I'm sure you can buy one of those "PHP in 21 days" books and get something that looks like results in 1-2 months, but that will be riddled with problems and security holes and will either break down the moment someone sneezes, or gets hacked and destroyed the moment someone looks at it funny. I've seen it before, it's not pretty.

jcbutler31
Posts: 3
Joined: Sat Aug 02, 2014 4:57 am UTC

Re: How to create an optimization engine

Postby jcbutler31 » Thu Aug 07, 2014 2:21 pm UTC

Tub wrote:This forum post is way too terse. How would the users' interests be modeled? Chosen from a predefined set of possible interests, freeform, ...? What features are you expecting that you haven't yet written down? Like the ability for your users to delete their accounts or reset their passwords?


I agree my post was too terse, I just didn't want to bore everyone with the specifics. Anyway, to your question, the user interests would be chosen from a predefined set of possible interests. For example, a user who is interested in sports would choose "Sports" as one of his or her interests. If a user is not a fan of all sports but just a fan of basketball, he or she would choose "Basketball" as a subcategory after clicking in the "Sports" tab. In the case of ESPN, they go even further in detail in that you can specifically specify the league, team, and/or university you are a fan of.

A user would be able to delete his or her account, and would also have the ability to reset a password. As far as other features, there would be a search option that a user could use to search certain topics or news events through the use of keywords. Also, the site would be accessible to all, even if you are not a registered for the site. Someone who is not registered for the site, obviously, cannot have the feature of a software-made optimized news article feed because they have not provided their interests. Non-members on the site would be able to access news by either choosing a topic on the title bar or searching for keywords that will link to a news article base for that keyword. These are traits found in nearly all news websites. How can I do all of this?

Tub wrote:Using existing software would free you from most of the technical burden so you can focus on creating content, attracting and keeping users.


Which software would be recommended to use?


As a final question, I would like to ask, if you were in my position and had to create a website like the one I've explained in this post along with my initial post, step by step how would you go about doing it and how long would your process take (considering I have very little, if any, coding experience and knowledge)?

Tub
Posts: 391
Joined: Wed Jul 27, 2011 3:13 pm UTC

Re: How to create an optimization engine

Postby Tub » Thu Aug 07, 2014 3:09 pm UTC

jcbutler31 wrote:I agree my post was too terse, I just didn't want to bore everyone with the specifics.

That's ok, I'm just saying you need to be very specific if you hire someone to program it for you, and include every detail, no matter if you think it's obvious. The functional specification needs to be part of the contract. If they don't know you want it, they won't include it in their estimate, thus they won't code it, and you'll end up with something you cannot use.

It's sad, but sometimes you need to include deliverables like "must actually work", "must not take 3 hours to satisfy a single page load", "must be secured against all known attacks", and (my favourite) "must not allow a user to login without supplying correct credentials". :roll:

jcbutler31 wrote:Anyway, to your question, the user interests would be chosen from a predefined set of possible interests. For example, a user who is interested in sports would choose "Sports" as one of his or her interests. If a user is not a fan of all sports but just a fan of basketball, he or she would choose "Basketball" as a subcategory after clicking in the "Sports" tab. In the case of ESPN, they go even further in detail in that you can specifically specify the league, team, and/or university you are a fan of.

Oh, so it's still a set of predefined tags, and you can use the existing tagging engine of any CMS system. Nesting is a bit more difficult, but I'd be surprised if that wasn't supported (native or via a plugin) in some of the existing systems. Selecting interests, having them saved, and automatically linking to the appropriate tag filter.. that might need some custom extensions.

jcbutler31 wrote:Which software would be recommended to use?

If I could recommend one, I would have done so. I'm one of those old-fashioned geeks who doesn't blog, twitter, facebook or any of these things that distinguish between writers and readers. I prefer communities amongst equals: forums, wikis, ... but those won't help you.

Wordpress seems popular, so that should probably be where you start your research.

jcbutler31 wrote:As a final question, I would like to ask, if you were in my position and had to create a website like the one I've explained in this post along with my initial post, step by step how would you go about doing it and how long would your process take (considering I have very little, if any, coding experience and knowledge)?

"had to"? Please tell me this is a hobby, not an attempt to make money. There are enough fools who think there's cash in copying existing, successful sites (just slightly different and with a fraction of the budget). Most of them fail.


I still don't know everything you want from the page. If I were to implement something like that from scratch, I'd guesstimate at least a man-month for initial development, split 50/50 between coding and graphics/layout/design (which would need another person, because I suck at anything graphics related). Estimates like these can easily double. I've had projects where my client kept getting these bright ideas "hey, that looks good, but it'd be even better if you did <X>", and all those little <X>s added up.
Plus ongoing maintenance: if the site grows, the users will want more features, and that takes more time. Coding from scratch means that there aren't any of the "nice-to-have" features an existing CMS might already have.


Starting with an existing CMS and an existing theme, it could be as easy as one afternoon for the setup. If nested tags aren't supported by the CMS, adding it would probably be between a day and a week. It's hard to say without taking a look at the CMS's code and architecture. These things are usually 100 hours of reading, 5 hours of trial and error coding, and in the end you have like 5 lines of code and it works. ;)

Those estimates are for my own work-speed. I cannot estimate yours.

User avatar
Xanthir
My HERO!!!
Posts: 5319
Joined: Tue Feb 20, 2007 12:49 am UTC
Location: The Googleplex
Contact:

Re: How to create an optimization engine

Postby Xanthir » Sat Aug 09, 2014 6:36 am UTC

It sounds like you're wanting to categorize/search *existing* news stories, not ones you wrote yourself, so ignore all the suggestions about Wordpress or tagging or the like. Those are all for creating and organizing your own content.

Doing this kind of thing on general web content is very difficult, requiring either some very smart algorithms, a bunch of relatively dumb manpower, or a combination of the two. Google News, for example, works partially on algorithms that read the articles and categorize them into topics based on powerful machine-learning algorithms that very smart engineers have honed over a long period of time.

This is not something you'll be able to duplicate, or even get near, with a small webdev team you hire on the cheap. It's something that'll cost $1M+, more than likely, unless you're the very smart person doing the work because you've had some new insight that you think will let you solve the problem more efficiently than it's been done before. Given that it sounds like you've just come up with it as a business idea, and don't have any deep technical insights, I must unfortunately tell you that there's next to no chance this can be done with anything approaching success.

People already have news sites, and unless you have a very compelling twist (and the ability to sign up for pre-determined categories is not a twist), you won't win them over. Solve some other problem, one that's annoying but not yet solved properly by anyone else.
(defun fibs (n &optional (a 1) (b 1)) (take n (unfold '+ a b)))

Tub
Posts: 391
Joined: Wed Jul 27, 2011 3:13 pm UTC

Re: How to create an optimization engine

Postby Tub » Sat Aug 09, 2014 3:50 pm UTC

Xanthir wrote:It sounds like you're wanting to categorize/search *existing* news stories, not ones you wrote yourself

Oh well, if that's actually what the OP wants, then I take it all back and agree with Xanthir. That's madness and not something that a single person with no formal education in computer science could do. For some reason that thought didn't occur to me because.. well.. it's madness. :roll:

User avatar
Jplus
Posts: 1711
Joined: Wed Apr 21, 2010 12:29 pm UTC
Location: Netherlands

Re: How to create an optimization engine

Postby Jplus » Sat Aug 09, 2014 8:09 pm UTC

It's worth mentioning that Newsblur does something very closely approaching what is being described in the OP, though. The only real difference is that it requires you to manually add newsfeeds (which will then be filtered by topics that you're interested in).
"There are only two hard problems in computer science: cache coherence, naming things, and off-by-one errors." (Phil Karlton and Leon Bambrick)

coding and xkcd combined

(Julian/Julian's)


Return to “Coding”

Who is online

Users browsing this forum: No registered users and 10 guests