So You Want to Write for Data Science Central

You're a writer on AI, ML, or various and sundry other data-oriented TLAs, and you'd like to write an article for Data Science Central. Great! This article is for you. Becoming a blogger on DSC is a good way to promote your proficiency in the field, to get the word out about interesting topics, or to gain the respect of your peers.

The mechanics of publishing on Data Science Central are straightforward:

• If you are not already a member of DSC, set up an account on Data Science Central or one of its related accounts (we're working on single sign-on, but we're not quite there yet).
• Wait for approval (you should receive an email letting you know when we add you to the membership roster.
• Once you are a member, log in, then select Home/Add Blog Post from the menu at top.
• You can use the WYSIWYG editor for content, or slip into HTML Editor mode. We hope to add support for Markdown and code display blocks early next year, but for now, you're somewhat limited in working with code.
• Save your draft with the Save Draft button. When you're ready to publish, select the Publish button.
• All content is moderated. This means that if the editor decides not to publish your article, they will not publish your article. If they do, then it will be published, typically within 2-3 days. If you have any questions, please contact the editors (including me).
• Data Science Central currently does not pay for content. On the other hand, the platform has more than half a million subscribers, so it is a fantastic place to post for exposure, and we do our best to promote content that we feel is worthwhile.
  
  

While the mechanics are important, it's also worth spending some time trying to understand what DSC is looking for in content:

• First up: Topics. When DSC was a brand new site, way back in 2012, the term Data Science itself was very novel, and it usually meant people who were able to use a new breed of programming tools (most specifically R, but later Python), to do analytics work, in the wake of the Big Data and Hadoop revolution that was going on at the time. Data Science Central was a cool, pithy name for the site, and as interest in the field grew, so did DSC.
• Coming up on a decade later, things have changed. Being a data scientist has overtaken programming as the wish list career topper that all aspiring nerds want to be when they grow up. Machine learning algorithms and convolutional neural networks are increasingly replacing traditional programming for a variety of activities, and data is becoming strategic within organizations rather than simply tactical.
• To that end, what we at DSC are looking for are stories about data. This can include data analysis tools and modeling, neural networks and data storage and access strategies, modeling, and knowledge representation. It also includes the strategic uses of data, governance, provenance, quality and protection, visualization and creative data story-telling. We're also expanding into those areas of artificial intelligence that are critical to cognitive computing, knowledge graphs, mathematics, and science. Why? Because data science is as much about science as it is about algorithms. Finally, DSC will focus more on the implications of data transformations on businesses, government, manufacturing, society and the individuals within it.
• We're looking for journalism. Some examples:
  • "How AI is transforming retail",
  • "Will GPT-3 win the Pulitzer Prize?",
  • "Data scientists and the political realm",
  • "Challenges of contact tracing in a post-COVID world",
  • "Penrose, Tiles and the Nobel Prize".
• We're looking for in-depth technical articles -
  • "How to digitally transform a company",
  • "AI in the Browser",
  • "Deep Fakes and the Algorithms That Drive Them",
  • "Knowledge Graphs for Publishing".
• We're looking for case studies
  • "Point of Failure: When AI Goes Rogue", "
  • Wrangling Drones",
  • "What happened to the Self-Driving Car?".
• Finally, we're looking for thought leadership -
  • "Where Do We Go From Here",
  • "Trolley Ethics",
  • "Who Really Benefits From AI?".
• We're looking for you to put on your teacher's hat, your prognosticator's hat, your analyst's hat, and tell us the about the world that YOU see.

All this being said, it's also worth understanding what we're NOT looking for.

• We are not looking for marketing pieces. If your product has an interesting toolset and you can dig into how to work with that toolset to solve complex problems, we might consider it, but we're more likely to send our sales-people to you to talk about ways that we can benefit one another (that's beyond of this editor's paygrade ... thankfully).
• We are not yet posting advertisements for jobs (or posting resumes, for that matter). This is not to say we aren't considering doing this, but like everything else, there's complexity in the implementation. If you have questions about either of these, feel free to contact the editors directly, and we can talk. We also have educational promotional products for universities and private institutions.
• Similarly, if you have events that you want to promote, talk to the editors. With the pandemic, we're awash in virtual seminars and conference notifications, but they do have value to the community.
• We occasionally do webinars and interviews, though in general this is likely to be something that we will handle directly. If you LIKE to interview others, either via video or digital print, please contact the editors.
• If you are the author of a book that you'd like to promote in the data science or knowledge engineering space, contact the editors.

Finally, a bit on style - things that will make your editors all tingly with delight, rather than awash in apathy.

• We prefer original content. If you have a paper on ArXiv, for instance,write up a story that summarizes the importance of that content, in more readable and less academic terms. If you want to repost elsewhere, you can do so, but we generally do not repost to existing articles not on the site unless they are exceptional, and that's usually at the editor's discretion, not the writer's.
• DSC is NOT a peer-reviewed journal. We welcome code and data samples (especially as we migrate to a new platform) but ultimately your audience is going to likely be technically proficient but not necessarily deep experts. As a rule of thumb - write to a tenth-grade audience, not a post-doctoral one. 
• We LIKE pictures. Diagrams, illustrations, photos, the whole worth a thousand words thing. However, if you do use pictures, make sure you have the rights to them. Our lawyers get unhappy when we have to speak with the other guy's lawyers. While on the current platform it's a bit awkward to do, we would also like to start including a splash graphic at the top of the article, primarily to generate thumbnails.
• Also, make sure you upload a good image of yourself when you are making your account. 
• In general, we prefer articles that run between about 600 and 1800 words.
• We're looking for professional writing - concise, easy to read, broken up into clear paragraphs.
• Identify your name, your title, and work or school affiliation.
• If you can, include a three to six bullet point list (called Data Points) covering the highlights or takeaways for the article itself. If these correspond with section headers, even better, but try to provide to your reader something to make them want to read your article.
• In general, DSC editors prefer fewer (or no) links to outside references, especially if they are promotional in nature. We also reserve the right to link to definition content on other TechTarget properties. If you do include links, try to make sure that they open in a separate pane (should be easier to do shortly), and in general such links should only appear at the bottom of the article, rather than inline (think footnotes)
• Include a short bio at the bottom of your article. You can link to a personal website or linked in page in the bio.
• Articles that are in draft form for longer than three months will be deleted.
• If you wish for an editor to review your comment and give you feedback, add an [Editor] tag at the bottom of the article with your questions. This will be deleted once the article is ready for publication.
• DSC makes no guarantees that it will publish an article once it has been submitted, though we will attempt to get back to you as quickly as possible about whether the article was accepted or not. 

Have fun, be creative, and take a chance. Welcome to Data Science Central.

Kurt Cagle
Community Editor
Data Science Central

from Featured Blog Posts - Data Science Central https://ift.tt/2HzDTjH
via Gabe's MusingsGabe's Musings

On the Nature of Data, Flights of Birds, and New Beginnings

My name is Kurt Cagle. I am the new Community Editor for Data Science Central, or DSC as it is known by its fans.

I'm one of those fans. Twelve years ago, Vincent Granville and Tim Matteson created a new site devoted to a passion they had: Data Science. In 2012, the term data science, and the practitioners, data scientists, were just beginning to come into vogue, specifically referring to the growing importance of a role that had been around for some time, the erstwhile data analyst, with the idea being that this particular role was different from a traditional programmer's role, though it borrowed many of the same tools.

Traditionally, an analyst, any analyst, has been someone who looks at information within a specific subject domain and, from their analysis, can both identify why things are the way they are and to a certain extent predict where those same things will be in the future.

Analysts have been around for a long time, and have always had something of a mystical air to them. As an example, in early Imperial Rome, there were a number of celebrated priests called Augurs who were said to be able to predict the future from the flight of birds. They had a surprisingly high success rate, and were usually in great demand by both military leaders needing strategic advice and merchants looking to better deploy their fleets and land agents. 

At first, the correlation between bird flight patterns and sound trade policy advice would seem low at best, but as with any good magical trick, it was worth understanding what was going on in the background. Why does one watch the sky for bird flight? Easy. Certain types of birds, such as homing pigeons, can carry messages from ships or caravans to various outposts, and from there such information can be relayed via both birds and other humans to central gathering points.  In other words, the Augurs had managed to build a very sophisticated, reasonably fast intelligence network tracking ships, troop movements, plague spots, and so forth, all under the cover of watching the skies for birds. Even today, the verb to augur means to predict, as a consequence.

In the eight years since Data Science Central published its first post, the field has grown up. Statistical and stochastic functions have become considerably more sophisticated. The battle royale between R and Python has largely been resolved as "it doesn't matter", as statistical toolsets make their way to environments as diverse as Scala, Javascript ,and C#. The lone data scientist has become a team, with fields as diverse as data visualization to neural network training to data storytellers staking their claims to the verdant soil of data analysis.

What is even more exciting is that this reinvention is moving beyond the "quants" into all realms of business, research, and manufacturing organizations. Marketing, long considered to be the least "mathematical" of disciplines within business, now requires at least a good grounding in statistics and probability, and increasingly consumes the lion's share of a company's analytics budget. Neural nets and reinforcement learning are now topics of discussion in the board room, representing a situation where heuristic or algorithmic tools are being supplemented or even replaced with models with millions or even billions of dimensions. The data scientist is at the heart of organizational digital transformations.

Let me bring this back to DSC, and give to you, gentle reader, a brief bio of me, and what I hope to be able to bring to Data Science Central. I have been a consulting programmer, information architect, and technological evangelist for more than thirty years. In that time I have written twenty-some-odd books, mainly those big technical door stoppers that look really good on bookshelves. I've also been blogging since 2003 in one forum or another, including O'Reilly Media, Jupiter Publishing, and Forbes. I spent a considerable amount of time trying to push a number of information standards  working with the W3C, and have, since the mid-2000s, focused a lot of time and energy on data representation, metadata, semantics, data modeling, and graph technology.

I'm not a data scientist. I do have a bachelor's degree in astrophysics, and much of a master's degree in systems theory. What that means is that I was playing with almost all of the foundational blocks of modern data science back about the time when the cutting edge processors were the Zylog-80 (known as the Z80) and 6502 chips within Apple II+ systems. I am, to put it bluntly, an old fart.

Yet when the opportunity to take over DSC came up, I jumped at it, for a very simple reason: context. You see, it's been my contention for a while that we are entering the era of Contextual Computing, eventually to be followed by Metaphorical Computing (in about twelve years, give or take a few). Chances are, you haven't heard the term Contextual Computing bandied about very much. It's not on Gartner's hype cycle, because it's really not a "technology" per se. Instead, you can think of contextual computing as the processing of, and acting on, information that takes place when systems have a contextual understanding of the world around them.

There are several pieces to contextual computing. Data Science is a big one. So is Graph Computing. Machine Learning, AI, the Internet of Things, the Digital Workplace, Data Fabric, Autonomous Drones, the list is long and getting longer all the time. These are all contextual - who are you, where are you, why are you here, what are you doing, why does it matter?

Data Science Central has become an authority in the world. My hope, my plan at this point, is to expand its focus moving into the third decade of this century. I'm asking you as readers, as writers, as community members, to join me on this journey, to help shape the nature of contextual computing. DSC is a forum to share technology but also to share asking deep questions about ethics and purpose, the greater good and with an eye towards opportunities. I hope to take Vincent and Tim's great community and build it out, with your help, observations, and occasional challenges.

from Featured Blog Posts - Data Science Central https://ift.tt/2F6e6yA
via Gabe's MusingsGabe's Musings

Free book - Artificial Intelligence: Foundations of Computational Agents

Free book - Artificial Intelligence: Foundations of Computational Agents

There are many excellent free books on Python – but Artificial Intelligence: Foundations of Computational Agents is about  a subject not commonly covered

I found the book useful as a introduction to Reinforcement Learning

As the title suggests, the book is about computational agents

An agent observes the world and carries out actions in the environment. The agent maintains an internal state that it updates. Also, the environment takes in actions of the agents, and in turn updates it internal state and returns the percepts. In this implementation, the state of the agent and the state of the environment are represented using standard Python variables, which are updated as the state changes.

This structure can be used to model many interesting problems and is the focus of the book. Ultimately, it leads to Reinforcement Learning.

The book structure is

Chapter 3: Searching for Solutions

Chapter 4: Reasoning with Constraints

Chapter 5: Propositions and Inference

Chapter 6: Planning with Certainty

Chapter 7: Supervised Machine Learning

Chapter 8: Reasoning Under Uncertainty

Chapter 9: Planning with Uncertainty

Chapter 10: Learning with Uncertainty

Chapter 11: Multiagent Systems

Chapter 12: Reinforcement Learning

Chapter 13: Relational Learning

The authors website also has detailed slides

I found the work exceptional so I bought the book Artificial Intelligence: Foundations of Computational Agents

But there is a free version

You can download the book – code and other resources at Artificial Intelligence: Foundations of Computational Agents

from Featured Blog Posts - Data Science Central https://ift.tt/3lauoFU
via Gabe's MusingsGabe's Musings

Data Detectives

It has become evident that developments in analytics are creating new occupations. There has been much discussion about where new jobs will come from with many existing ones being made redundant because of the 4^th Industrial Revolution – i.e. the impact of artificial intelligence and robotics. Analytics is bucking this trend.

 

Some of new occupations in analytics include data prospectors and data harvesters. Data prospectors, like gold prospectors, are responsible for searching and locating data on the internet and other large data repositories. Data harvesters are responsible for extracting data and information from these sources. Data harvesters do this, for example, by web scraping. Staff who are highly skilled and knowledgeable in doing these functions are required - especially exploring something as vast and intricate as the internet.

 

Another new occupation is that of a data detective. They are analysts who find knowledge and insights in data. This may sound a simple and straight forward job to do.

 

It is suggested that there are plenty of analysts who can do extraction and cleaning tasks but have little or no aptitude for exploring data to find answers to difficult problems and issues and struggle to recognise important and informative discoveries. That is, they can perform the technical tasks of providing data but are not able to use it to find ‘nuggets of gold’ in this resource.

 

What is required are highly skilled professionals who, like police detectives, excel at analysis and problem solving. They need to be proficient in marshalling facts, following leads in data, testing hypotheses and hunches, joining the ‘dots’ and drawing conclusions from what is known. In short, they require the knowledge and skills of a Sherlock Holmes.

 

The primary skills required by data detectives are the ability to explore data and the ability to identify items of interest. They can do this by using the functionality of desktop packages such as Microsoft Excel and Microsoft Access and data visualization packages such as Tableau, QlikView and Power BI. They can also interrogate data using SQL with structured data and SPARQL with semantic data.

 

Where data detectives add value is that they ask informed questions to help to understand challenging and difficult problems and issues. They find workarounds when they hit difficulties and obstacles in obtaining the answers they require. They possess the nous, have the patience, and have the persistence to go the extra kilometre to find interesting patterns and trends in data.  

 

Three examples of where data detectives can add value include using risk-analysis tools to gain insights into threats and opportunities. They can take different data views of subjects and issues and where interesting patterns are found, they can make further inquiries to find more about what is going on and what their implications are when it comes to developments that can either harm or benefit individuals, organizations and the community.

 

The second example is stratifying a population to find interesting strata such as those with high incidence of a disease such as COVID-19. They can analyse cases in different strata to see why they have high infection rates and compare these with strata with low infection rates. These analyses can reveal what measures can be taken to lower the incidence of the disease.

 

The third example is analysing cases that have anomalies with insurance claims. Business rules can be written for those who show unusual patterns and the rules can be cascaded to find other people in the population who closely match them as they too may have issues with their claims.

 

It is suggested that data detective work needs to be recognized as a specialist skill where those with requisite attributes are selected, trained, and employed to do this work. Organizations need to take steps to identify those who are gifted in doing detective tasks and use their talents.

 

They complement data scientists who use mining and modelling techniques to extract knowledge from data. Data detectives are more qualitative in their approach while data scientists are more quantitative in their orientation. However, data detectives use the tools and procedures developed by data scientists to explore data such as using population partitioning techniques.

 

Data detectives can go the extra step of interpreting what data scientists find in data and can give context to what is discovered or detected.  For example, data scientists can produce a list of high-risk cases detected using a machine-learning model but often they cannot explain why they are classified in this manner. Data detectives can explore data to give context to the cases and explain why they were identified by the model. They can also spot false positives or cases that appear to be of concern but are false alarms and therefore do not warrant attention. This saves time, money and effort in that resources are not wasted pursuing them.   

 

Data detectives are part of the broader and growing family of occupations that deal with data. This family includes as examples data prospectors, data harvesters, data scientists, data analysts, data engineers, data architects, data brokers, data lawyers, data journalists, data artists, data quality officers and database managers. They each have a discrete and important role to perform and they all complement each other in making use of what is now referred as the new oil. Data is now the fuel that enables organizations to function and to deliver business outcomes.  

 

When it comes to formal education, there are now many masters programs in analytics in universities across the globe. These programs could be expanded to include different specialization streams to cater for these different data occupations cited above. That is, they become omnibus programs where students can select relevant subjects that enable them to specialize in data science or data engineering or data brokering or data detective work to use examples. These specializations are required to provide practicing analytics professionals to meet the diverse needs of government, industry, and commerce in the 21^st Century.

 

 

Bio

Warwick Graco is a practicing data scientist and can be contacted at Warwick.graco@analayticsshed.com

from Featured Blog Posts - Data Science Central https://ift.tt/2F01TeB
via Gabe's MusingsGabe's Musings

Introducing Analytics To A Product

With more and more people getting conversant in analytics, its demand in every field becoming more pronounced. Product is no different. It is almost inevitable to introduce analytics in some format or other in the products you own. However, before you open the gates to this world of magic, there are three questions you should try answering.

These three basic questions shall help in better planning for your analytics strategy and would act as a compass in times of uncertainty.

 

Why analytics:

Great that you have decided to embark on the journey - could be because of fear of missing the bandwagon. Nevertheless, without answering this question, your team would always be involved in directionless busy work. It relates to a simple finding of why you want to introduce analytics to product. As Mckinsey puts it, without the right question, the outcome would be marginally interesting but monetarily insignificant.

1. Is it to enable end users of your product?
2. Will it serve for your internal product intelligence?
3. Is it because every other product has some flavor of analytics?
4. Investors asking for it?
5. Is it the next big strategy for the product roadmap?
6. You have hired a data science team; you do not know what to do with?
7. Would it provide a better selling proposition for the sales team?
8. If your clients are asking for their usage statistics?

What in analytics:

Isn’t it obvious to ask yourself what you want to build, before you actually start building. Similar is true for analytics.

1. Do you want to enable reporting of various metrics for admins of your B2B platform?
2. Would you leverage AI/ML for a product feature for end users?
3. Are you looking for more in-depth product intelligence?
4. Is it a good to have feature, without much usability? OR it is going to be the prime feature offering?

 

Answer to the above questions, is a function of the product type, its intended use and the users.

How to analytics:

After addressing Why & What to offer in analytics, the logical next step is to plan how to deliver it. Moreover, when we are talking about analytics, data is the centerpiece. Data enforces the need of a completely new ecosystem of processes and practices to meet the regulatory, trust and demand obligations.

Although, every constituent of data management calls for a dedicated article, I shall touch briefly on each and try to illustrate how each influences the analytics strategy of the product.

Database systems: The primary infrastructure that would act as the cornerstone of the analytics strategy: database to store the data. RDBMS, NoSQL, or a Hybrid solution, followed by dozens of companies to choose from.

Master data and metadata management: This is the definition, the identity, the identifier, the reference via which every data call will be directed. It is essential to know and govern extensive data assets.

Quality control: You must have heard of the saying ‘garbage in, garbage out’. Bad data will severely hamper the trust and actionable knowledge in business operations. Data has to be unique, complete and consistent.

Integration definition: For analytics to be practical and actionable, data has to flow in from varied sources. This can be a transfer between different products or join between multiple modules within the platform. A schema is a map or viaduct that enable this unification.

Warehouse: The transactional data or raw data stored from platform might not be ideally designed for analytics. Joining a dozen of tables on the fly would impact not just the throughput but also the very feasibility of insight generation. A purpose built data warehouse is an efficient step towards integrating data from multiple heterogeneous sources. However, this may lead to a near-real-time system with some delay in data availability.  

Transformation: Data transformation is an integral part of data integration or data warehousing, where the data is converted from one format/structure to other. It involves numeric/date calculation, string manipulation or rule based sequential data wrangling processes. As a step in ETL (extraction-transformation-load) data transformation cuts down the processing time for end user, thus enabling swift reporting and insight generation.

Governance: Sets the guiding principles, benchmarks, practices and rules for 1) Data policies 2) Data quality 3) Business policies 4) Risk management 5) Regulatory compliances 6) Business process management. Being an essential part of RFPs and government regulations, lack of data governance can expose company to lawsuits, higher data/process costs and complete business failure.

Architecture: According to the Data Management Body of Knowledge (DMBOK), Data Architecture “includes specifications used to describe existing state, define data requirements, guide data integration, and control data assets as put forth in a data strategy.” Simply said data architecture describes how data is collected, stored, transformed, distributed and consumed. Data architecture bridges business strategy and technical execution.

 

Without the power to derive of information and insights, storing data is of no use. Once the data management is in place, planning is required for processing and representing the data.

Collaboration vs in-house development: There are tons and tons of tools available in market that help making sense out of your data. These can be traditional BI tools like Power BI/ Tableau/ Qlik/ Microstrategy that help make dashboards. Or, there are modern BI tools like Looker/ Periscope/ Chartio which go beyond just dashboarding. Then there are tools like Amplitude/ Firebase/ Google Analytics/ Mixpanel/ Moengage which help with product analytics and understanding user behavior. These tools easily integrate with your product and provide faster go-to-market for your analytics offering. However there is cost associated with this – 1) steep recurring subscription cost 2) lesser control on features. An alternative could be developing the reporting and dashbaording tool in-house. It does come with a very long gestation/development period and operational issues of larger teams to manage. However, these can be off-set by the cost savings and superior control.

Real time or delayed: With business needs driving this decision, comparison could be made between consuming transactional data (real time analytics) or warehouse data (near real time) for analytical purposes. A warehouse definitely has an edge, providing more flexibility and scope but real time reporting has its own charm.

AI/ML: Artificial intelligence and Machine learning are the latest buzzwords, with something as humble as macro automation being classified as AI/ML. However, the sincere AI/ML solutions enable the product a proposition of differentiated offering along with the essential value add for the end users. The only concern with AI/ML implementation is that of cost. Whether human talent or infrastructure, it does not come cheap. Not to forget, the much-needed patience and trust that needs to be invested by the leaders. Hence, the agreement has to be a well thought though business decision, rather than a hasty push from IT department.

 

Essentially,  rather than a knee-jerk reaction, a well thought out plan – considering demand, capabilities, resources, company’s management, business, legal and regulations – ensures the analytics implementation a definite success.

from Featured Blog Posts - Data Science Central https://ift.tt/30zrC59
via Gabe's MusingsGabe's Musings

GPT3 and AGI: Beyond the dichotomy – part two

This blog continues from GPT3 and AGI: Beyond the dichotomy – part one

GPT3 and AGI

Let’s first clarify what AGI should look like

Consider the movie ‘Terminator’

When the Arnold Schwarzenegger character comes to earth – he is fully functional. To do so, he must be aware of the context. In other words, AGI should be able to operate in any context

Such an entity does not exist

And nor is GPT3 such an entity

But GPT3 however has the capacity to respond ‘AGI-like’ to an expanded set of contexts much more than traditional AI systems.

GPT 3 has got many things going for it

• Unsupervised learning is the future
• Linguistic capabilities distinguish humans
• But Language is much more than encoding information. At a social level, language involves joint attention to environment, expectations and patterns.
• Attention serves as a foundation for social trust
• Hence, AGI needs a linguistic basis – but that needs attention and attention needs context. So, GPT-3 – linguistic – attention – context could lead to AGI-like behaviour

Does AGI need to be conscious as we know it or would access consciousness suffice?

In this context, a recent paper

A Roadmap for Artificial General Intelligence: Intelligence, Knowledge, and Consciousness: Garrett Mindt and Carlos Montemayor

https://www.academia.edu/43620181/A_Roadmap_for_Artificial_General_Intelligence_Intelligence_Knowledge_and_Consciousness makes an argument is that

• integrated information in the form of attention suffices for AGI
• AGI must be understood in terms of epistemic agency, (epistemic = relating to knowledge or the study of knowledge) and
• Eepistemic agency necessitates access consciousness.
• access consciousness: acquiring knowledge for action, decision-making,  and  thought,  without  necessarily  being  conscious

 

Therefore, the proposal goes that AGI necessitates 

• selective attention for accessing information relevant to action,  decision-making,  memory  and    
• But not necessarily consciousness as we know it

 

This line of thinking leads to many questions

• Is consciousness necessary for AGI?
• If so, should that consciousness be the same as human consciousness
• Intelligence is typically understood in terms of problem-solving. Problem solving by definition leads to specialized mode of evaluation. Such tests are easy to formulate but check for compartmentalized competency (which cannot be called intelligence). They also do not allow intelligence to ‘spill over’ from one domain to another – as it does in human intelligence. 
• Intelligence needs information to be processed in a contextually relevant way.
• Can we use epistemic  agency  through  attention as the distinctive mark of general intelligence even without consciousness? (as per Garrett Mindt and Carlos Montemayor)
• In this model, AGI is based on joint attention to preferences in a context sensitive way.
• Would AI be a peer or subservient in the joint attention model?

Finally, let us consider the question of spillover of intelligence. In my view, that is another characteristic of AGI. Its not easy to quantify because tests are specific to problem types currently. A recent example of spillover of intelligence is from facebook AI supposedly inventing it’s own secret language. The media would have you believe that groups of AGI are secretly plotting to take over humanity. But the reality is a bit mundane as explained. The truth behind facebook AI inventing a new language

In a nutshell, the system was using Reinforcement learning. Facebook was trying to create a robot that could negotiate. To do this, facebook let two instances of the robot negotiate with each other – and learn from each other. The only measure of their success was how well they transacted objects. The only rule to follow was to put words on the screen. As long as they were optimizing the goal(negotiating) and understood each other it did not matter that the language was accurate (or indeed was English). Hence, the news about ‘inventing a new language’. But to me, the real question is: does it represent intelligence spillover?

Much of future AI could be in that direction.

To Conclude

We are left with some key questions:  

• Does AGI need consciousness or access consciousness?
• What is role of language in intelligence?
• GPT3 has reopened the discussion but still hype and dichotomy (both don’t help because hype misdirects discussion and dichotomy shuts down discussion)
• Does the ‘Bitter lesson’ apply? If so, what are its implications?
• Will AGI see a take-off point like Google translate did?
• What is the future of bias reduction other than what we see today?
• Can bias reduction improve human insight and hence improve Joint attention?
• GPT-3 – linguistic – attention – context
• If context is the key, what other ways can be to include context?
• Does problem solving compartmentalize intelligence?
• Are we comfortable with the ‘spillover’ of intelligence in AI? – like in the facebook experiment

 

References

https://towardsdatascience.com/gpt-3-the-first-artificial-general-intelligence-b8d9b38557a1

https://www.gwern.net/GPT-3

http://haggstrom.blogspot.com/2020/06/is-gpt-3-one-more-step-towards.html

https://nordicapis.com/on-gpt-3-openai-and-apis/

https://www.datasciencecentral.com/profiles/blogs/what-is-s-driving-the-innovation-in-nlp-and-gpt-3

https://bdtechtalks.com/2020/08/17/openai-gpt-3-commercial-ai/

https://aidevelopmenthub.com/joscha-bach-on-gpt-3-achieving-agi-machine-understanding-and-lots-more-artificial/

https://medium.com/@ztalib/gpt-3-and-the-future-of-agi-8cef8dc1e0a1

https://www.everestgrp.com/2020-08-gpt-3-accelerates-ai-progress-but-the-path-to-agi-is-going-to-be-bumpy-blog-.html

https://www.theverge.com/21346343/gpt-3-explainer-openai-examples-errors-agi-potential

https://www.theguardian.com/commentisfree/2020/aug/01/gpt-3-an-ai-game-changer-or-an-environmental-disaster

http://dailynous.com/2020/07/30/philosophers-gpt-3/

https://marginalrevolution.com/marginalrevolution/2020/07/gpt-3-etc.html

https://artificialintelligence-news.com/2020/09/10/experts-misleading-claim-openai-gpt3-article/

https://analyticsindiamag.com/gpt-3-is-great-but-not-without-shortcomings/

https://www.3mhisinsideangle.com/blog-post/ai-talk-gpt-3-mega-language-model/

https://venturebeat.com/2020/06/01/ai-machine-learning-openai-gpt-3-size-isnt-everything/

https://discourse.numenta.org/t/gpt3-or-agi/7805

https://futureofintelligence.com/2020/06/30/is-this-agi/

https://www.quora.com/Can-we-achieve-AGI-by-improving-GPT-3

https://bmk.sh/2020/08/17/Building-AGI-Using-Language-Models/

https://news.ycombinator.com/item?id=23891226

 

image source: Learn English Words: DICHOTOMY

from Featured Blog Posts - Data Science Central https://ift.tt/2RZXiMq
via Gabe's MusingsGabe's Musings

Weekly Digest, September 21

Monday newsletter published by Data Science Central. Previous editions can be found here. The contribution flagged with a + is our selection for the picture of the week. To subscribe, follow this link.  

Announcement

• How to Automate your Cloud Data Warehouse - Upcoming DSC Webinar

Featured Resources and Technical Contributions 

• Stumped by Bayes' Theorem? Try This Simple Workaround
• Data-driven innovation in healthcare: synthetical clinical data 
• All-Purpose Universal Platform for Real-Time AI/ML +
• 8 Smart Ways To Become A Data Scientist
• A Programming Guide with Probability and Statistics
• CALL SYMPUT in SAS Explained with Examples
• New Coursera Series: Machine Learning for Everyone
• Monitoring the Health of Any Data

Featured Articles

• Risk Avoidance Spectrum and Character Types
• Algorithms of Social Manipulation
• 3 Big Businesses Must Adopt DevOps and Cloud Computing
• The illusion of choice
• IoT’s Purview on Current Environmental Conditions
• Difference Between Deep Learning and Machine Learning in one Infographics
• Difference Between Correlation and Regression in Statistics
• Deep Learning Explained in 4 Simple Facts

Picture of the Week

Source: article flagged with a + 

To make sure you keep getting these emails, please add  mail@newsletter.datasciencecentral.com to your address book or whitelist us. To subscribe, click here. Follow us: Twitter | Facebook.

from Featured Blog Posts - Data Science Central https://ift.tt/35SmX1G
via Gabe's MusingsGabe's Musings

The illusion of choice

Do you think your actions are the result of your own free choices? What if those actions are the inevitable and necessary consequence of antecedent states of affairs? What does this mean for your free will?

In a deterministic world where there’s an exclusive future for all our actions, digital users can become more predictable and monetizable than ever. In fact, by using creative designs and deceptive strategies, companies can create deterministic worlds and exploit the fact that human behaviour is hardwired to choose the path of least resistance.

Sites and apps designs become highly relevant in this scenario, because if you know how people think, you can design choice environments that make it easier for people to choose what you want. Several companies observe and learn an incredible amount of information about user behaviour in order to refine what is called “choice architectures,” discrete design elements intended to influence human behaviour through how decisions are presented.

There are choice architectures all around you, and they are never neutral: they always influence user behaviour, even when they fail to accomplish its objective or there’s no explicit strategy behind.

Nudging

“Nudging” refers to how users can be driven towards making certain choices by appealing to psychological biases. A nudge is any aspect of the choice architecture that alters people’s behaviour in a predictable way without forbidding any options or significantly changing their economic incentives. It represents a small feature of the environment that alters people’s behaviour but does so in a non-enforced way.

It’s true that choice architectures influence by default (even when there’s no apparent intention behind), so a nudge is best understood as the intentional attempt at influencing choice.

Example of nudging: the pre-selected amount suggests is the ‘right’ one. Source: The Internet Patrol

Nudges work without limiting the user’s original set of choices, and can produce positive results like:

• Improving health caring. Sending people a simple reminder to schedule a dental check-up proved to double the rate of people who signed up for an appointment.
• Improving financial decisions. Sending students a few personalized text messages helped many of them remember to refile their application for student aid.
• Increasing choices that benefit others. Using an opt-out system for organ donations where people are automatically registered as organ donors can significantly increase the number of donors.

Whether through reminders, personalized notifications, awards or default settings, nudges can steer people to make better choices. But the question is: better to whom? What if there were opposite interests behind the act of driving user’s behaviours?

The dark side

There are tricks that go beyond nudges to influence decision making, and cause users to do things they may not otherwise do. Dark patterns are subtle ploys many companies use to manipulate you into doing something, such as buying or signing up for something, or disclosing personal or financial details.

A design that delivers the best conversion rate might not be the same that delivers the best user experience.

Example of a dark pattern applied to hide extra costs. Source: Shopify

Dark patterns may exploit timing to make it harder for a user to be fully in control of its faculties during a key decision moment. In other words, the moment when you see a notification can determine how you respond to it. Or if you even notice it.

Dark patterns also leverage on a very human characteristic: we’re lazy by design, so producing friction is always an effective strategy. Designs that require lots more clicks/taps and interactions discourage users into engaging with that content.

In this example, the user must actively unselect the extra product, or otherwise it will appear in the checkout. Source: Econsultancy

The site Darkpatterns.org details some of these deceptive mechanisms like “roach motel” (when the design makes it simple to sign up but hard to cancel), “disguised ads” (ads masqueraded as content that isn’t trying to sell you something), or “privacy Zuckering” (named after Facebook CEO Mark Zuckerberg, is the trick of getting you to overshare data).

These strategies extend everywhere on the internet, and although some countries try to regulate this behaviour, the truth is that not all governments seem to be taking action:

• Google has been accused of using manipulative design practices to trick users into clicking on ads, making it nearly impossible to tell ads apart from organic search results.
• TurboTax is a US tax preparation software to file taxes online that has been accused of making it ridiculously complicated to find the actual free part of the service, and even making sure to hide it from Google results.
• Microsoft has also been accused of using dark patterns to push users into opening cloud accounts, making the offline-account option harder to find.

As digital users, we are in a very hard spot. The fact that many US banking sites are harder to read than Moby-Dick (making 58% of US bank content not readable for the average American), and about 11% of retail websites contain dark patterns, reveals that we are continuously under siege. Is there any way out?

Final thoughts

Manipulation through behavioural techniques can occur quietly and leave no trace. Since companies can drive customer’s decisions through heavy analytics and user interfaces, it’s easy to imagine a digital future in which social platforms employ algorithms to predict the virality and monetizability of each post, only accepting or highlighting the ones that could generate sufficient revenue.

Companies are super focused on testing and experimenting with different techniques to get the most desirable responses, and since they are incredible experts in the discipline, it seems hard to avoid being deceived. The good news is that education is a powerful tool, and by knowing some of their strategies and trying to be aware of our cognitive biases, we might be able to sidestep some of the traps.

Sites like Darkpatterns.org or Princeton’s Web Transparency have lots of examples regarding dark patterns and awesome material. I suggest you take some time and go through them. It will pay off. You can also explore @darkpatterns or use the hashtag #darkpatterns on Twitter to call out what you’ve found or discover what others have found. Social media engagement is a good way to put pressure on companies to stop using these practices.

If you can detect deceptions, you’re more likely to avoid them

Interested in these topics? Follow me on Linkedin or Twitter

from Featured Blog Posts - Data Science Central https://ift.tt/3mydQJz
via Gabe's MusingsGabe's Musings

8 Smart Ways To Become A Data Scientist

Data Science has been coined everywhere, and everybody wants to express their views and thoughts on this subject even though they are a novice or lack adequate knowledge about data science! The idea that everyone can become a data scientist only by studying a few technological advances and solving any complicated problems cripples the world today. 

Let us first understand the career perspectives of data science. Data science has three pillars:

1. Business Expert - Data Analyst
2. Technology Expert - Data Engineer
3. Statistics & Algorithm - Data Scientist

First, make it clear whether you want to become a data analyst, a data engineer, or a data scientist.

Smart ways to become a data scientist

• Education requirement

Education is known to be one of the primary sections of resumes and it is not going to change at all. Educational background serves a signal to the employers to better know about their future employees. When it comes to Data Science, you’ll find most of the professionals holding Ph.D. education. As per the data gathered by 365datascience, the typical data scientist in 2020 holds a Master’s degree (56%), a Bachelor (13%), or a Ph.D. (27%) as their highest academic qualification. The highest level of education achieved by data science professionals is a doctorate. Though, the considerable drop in being a data scientist is a bachelor-level degree only. The advanced levels are just to ensure a specialization in data science. We have discussed degrees! But what is the best degree to become a data scientist? To answer this, degree related to Computer Science or Statistics and Mathematics are inclined to data science proficiency. Data Science and Analysis graduates have made their room on the top of the research on the career path of becoming a data scientist.

Let me tell you a hack here! In reality, no single degree can prepare a person for a real job in data science. Even if you are post-graduate in data science, but you don’t have strong analytical and programming skills, you can’t be a data scientist.

• Learning formats

Primarily, you can choose either of the three learning formats- Online training, Offline training, and Self-learning. You can avail of data science training from various platforms like Coursera, Deakin, Udemy, and many more. Offline training, on the other hand, can also be a great option if you have proper resources of data science learning available around you. If you are already in the profession and don’t have time to avail of both online and offline learning, then you have another option to explore, it is self-learning. What you need is to religiously follow various resources online and offline. You can subscribe to various YouTube channels providing data science training and watch the videos as per your time availability.

• Hands-on learning

Solving real-world data science problems will only improve your practice in data science. But from where will you start? Working with the dataset with the classic Titanic data set with survival classification or clustering is likely to damage your portfolio, rather than help. Instead, consider taking ideas from shared Github ventures. Look at what others are creating based on the network that you acquired from LinkedIn through tech sessions and certifications. Feel free to use samples from Github projects on Udacity or Coursera. Then mix real datasets from Google Research, Kaggle, or search for an interesting dataset and start building real problem solutions. If you are involved in a particular field or business, consider looking for public datasets and developing a sample project. For example, if you're interested in fintech, try building a loan approval algorithm using the public loan data from Lending Club. The key takeaway to work with real datasets is that they are very messy and noisy compared to academic ones. You need to prepare the skillset by practising online datasets available. What all you need to do is:

• Download and open the data in Excel or a related application 
• See what trends would you find in the data by eyeballing them 
• If you think the evidence-backed the article's conclusions? Whether or not? 
• What more questions do you think you should answer with the data?

• Programming skills

Starting up in programming can be very challenging, and there are a lot of myths out there that make people think programming is a skill they can never learn, or that landing a job as a data scientist is almost mission impossible. They couldn't be any worse. To become a Data Scientist, prepare yourself to master the following skills:

• Statistics
• R and Python programming language
• Data wrangling
• Mathematical skills like calculus and linear algebra
• Data visualization
• Visualization tools like a tableau, ggplot, plotly.
• Predictive modelling
• Machine learning

Research shows that the sector is continually changing and adapting to market needs as well as its rising importance in academia and around. Universities are meeting demand while Master's is defining itself as the traditional golden degree. Python keeps consuming away at R, but even SQL is on the rise!

• Algorithmic approach

Your prime emphasis should be on a deep understanding of the algorithms. You should be able to answer certain questions as-

• What are the input and outputs of the algorithm?
• What are the assumptions that are part of the algorithm?
• Where does it fail?
• And the master question to test your expertise. If given time and resources, can you manually run an algorithm with just a pen and paper?

If you can answer these for any algorithm then you have acquired experience for that algorithm at the data science level. Practice makes a man perfect is a right saying to be quoted here!

• Stop believing myths

Whilst on the learning path of data science, you might get to know about several myths. My recommendation to you here is DON’T BELIEVE ANY MYTHS, I repeat, DON’T BELIEVE THEM AT ALL! Rather, focus more on:

• Data science is about being able to answer questions and generate value for business, not software 
• Learning the definitions matter more than learning the syntax 
• Creating and sharing projects is what you are going to do in an actual data science job, and practicing this way will give you a head start

• Practice on storytelling with data

A story in the sense of data science is a tale of what you've discovered, how you find it, and what it means. An explanation could be the revelation that in the past year the company's sales have fallen by 20 percent. It's not enough to merely state the fact — you're going to have to explain why sales fell, and how to address it. The key components of data-storytelling are: 

• Understanding and contextualization 
• Find ways to investigate 
• Using imperative data visualization 
• Applying various data sources 
• Have a coherent narrative

• Supplementing data

In your data scientist job, you will be given raw datasets only. So, you should have the ability to combine raw datasets before performing data analysis. The first move in creating a supplementing a good quality dataset is to know what expertise to show. The key skills businesses seek in data scientists, and therefore the primary skills they want to show in a portfolio, are: 

• Control of communication 
• Power to work with others 
• Technical know-how 
• Power to reason with data 
• Motivation and the potential to take action

Are you a future-ready Data Science professional?

Learning data science will be time-consuming – say 3 to 12 months or more of regular learning is required. By demonstrating a higher demand for junior data scientists than the US and the UK, India has won the position of the best country to start a career as a data scientist. If you just have a Bachelor's degree, it is still the place to be. You just began a life-long adventure, which promises exciting experiences to enjoy. So keep your interest refreshing, develop your collection of programming skills, and good luck in your career in data science! The ball is in your court!

from Featured Blog Posts - Data Science Central https://ift.tt/35SIxmJ
via Gabe's MusingsGabe's Musings

CALL SYMPUT in SAS Explained with Examples

CALL SYMPUT and CALL SYMPUTX

CALL SYMPUT and CALL SYMPUTX in SAS are the most commonly used DATA Step call routines to interact with the Macro Facility in SAS. Both are used to assign a value of DATA step to a macro variable.

SYNTAX:

CALL SYMPUT(macro-variable, value);

Arguments are macro-variable and can be one of the following items:

• a character string that is a SAS name, enclosed in quotation marks.
• It can contain the name of a character variable.
• A character expression to create a macro variable name. You can create a series of macro variables using this form.

value is the value to be assigned, which can be

• a string enclosed in quotation marks.
• It can be the name of a numeric or character variable. The macro variable is assigned with the value of the variable,
• a DATA step expression. The value returned by the expression in the current observation is assigned as the value of macro-variable.

Example 1: Using CALL SYMPUT to create macro variables of DATA step variables.

This DATA step creates the three macro variables countries, and respectively assign them the values in the code variable.

data currency_codes;
input country $ 6. code $4.;
call symput(country,code);
datalines;
USA USD
UK GBP
Japan JPY
Europe EUR
India INR
;
run;
%put &India;

OUTPUT

INR

Example 2: Creating a series of macro variable

In the below example, the CALL SYMPUT statement builds a series of macro variable names by combining the character string Country and the left-aligned value of _N_. Values are assigned to the macro variables Country1, Country2, and so on.

data currency_codes;
input country $ 6. code $4.;
call symput('Country'||left(_n_),country);
datalines;
USA USD
UK GBP
Japan JPY
Europe EUR
India INR
;
run;
%put &country2;

OUTPUT

UK

Below is an example of a string enclosed in quotation marks  the statement assigns the string testing to the macro variable NEW:

data test;
call symput('new','testing');
run;
%put &new;

OUTPUT

testing

Example- When the value is the name of a numeric or character variable.

The macro variable is assigned with the value of the variable,

For numeric variable, SAS performs an automatic numeric-to-character conversion and writes a message in the log.

data test;
a=2;
b="a character variable";
call symput('a',a);
call symput('b',b);
run;

Use this form when macro-variable is also the name of a SAS variable or a character expression that contains a SAS variable.

A unique macro variable name and value can be created from each observation using the automatic data step variable _n_. See Example 2.

For character variables, SYMPUT creates only one macro variable, and its value changes in each iteration of the program. The macro variable contains only the value assigned in the last iteration of the data step.

data class;
set sashelp.class;
call symput('Name',name);
run;
%put &name;

OUTPUT

William

Example when the value contains a  DATA step expression.

The value returned by the expression in the current observation is assigned as the value of macro-variable. In this example, the macro variable named HOLDATE receives the value July 4, 1997:

Also, Read the Date Interval Functions – INTNX and INTCK in SAS

from Featured Blog Posts - Data Science Central https://ift.tt/32L7f6u
via Gabe's MusingsGabe's Musings

3 Big Reasons Every Business Must Adopt DevOps and Cloud Computing

Cloud computing has become ubiquitous in today’s fast-paced business environment. When you combine this dynamic technology with DevOps culture, your business transforms in real-time. With benefits like on-demand scalability, quicker run times, consistent codes, and high-performing apps, this game-changing combination can empower your business to become more agile, responsive and swift.

If you haven't implemented it yet, here are three big reasons to do it right away:

1. Accelerated Time to Market: Automation is a key driving factor for infrastructure management. It determines the success of a product or service by accelerating its time to market. When business processes are powered by cloud computing, they can be made more efficient and error-free with automation. With a reliable infrastructure, developers can deliver high-performing solutions faster. Automation plays a vital role in optimizing IT processes and enabling efficient management. With the cloud, you can automate processes like running test cases, provisioning, or compiling reports from collected data.
2. Faster Deployment: When the cloud powers IT processes, companies can deploy the code faster. However, customized and error-free deployment requires that you use DevOps in Cloud. Businesses that are backed by a DevOps culture can solve their IT issues by building custom logic. With easy access to the latest and advanced tools, it also becomes easier to define capabilities for better business outcomes. 
3. End-to-End Monitoring: Cloud makes it possible to keep all your essential tools for crucial projects in one centralized platform. From monitoring to backup to automation tools, everything is readily available in one place. This easy access to all infrastructural services is one of the most significant benefits of Cloud. It empowers developers to ensure the success of software solutions.

DevOps in Cloud can be leveraged for managing and monitoring the latest versions of all the tools. Companies can also set up custom alarms and notifications. These alerts are triggered to optimize tools for efficient use.

Backup for Testing

Every cloud service provider has a backup mechanism to tackle unexpected performance issues. Despite this backup, the absence of DevOps can invite automation issues in applications. When this happens, servers need to be launched manually for restoring the backup. Such cases can hurt the end-user experience, causing customers to lose faith in the reliability of your business.

Conclusion

It is cloud computing that empowers businesses to unlock their full potential. It enables you to undertake load testing and assess the reliability of apps. Cloud computing also makes mobile automation testing possible. It integrates advanced tools to recreate the software development environment.

But, if these capabilities are used without DevOps in Cloud, replicating the production environment for testing can become a complicated process that is prone to errors.

from Featured Blog Posts - Data Science Central https://ift.tt/3ciRw1W
via Gabe's MusingsGabe's Musings

Algorithms of Social Manipulation

Do you know how your apps work? Are you aware of what tech companies are doing in the back with your data? And what’s more revealing: do you know which of your action are actually influenced by those apps? When you take a trip with Uber, buy stuff on Amazon, or watch a movie on Netflix: when are you consciously deciding and when are you being heavily influenced?

Tech companies are not passively observing your behavior and acting as a consequence: they are influencing your behavior so you become more predictable. By conditioning your actions, companies can predict outcomes in a better way, and know better what to sell you.

Every breath you take

Systems and apps make use of massive amount of data like user’s location, maps, browser’s interests, and data streams coming from mobile and wearable devices, in an era of unprecedented power for firms who are no longer merely appealing to our innate desires, but programming our behaviors.

The site True People Search probed the privacy policies of 11 of the biggest tech companies in the world to find out exactly what they know about us, and the result is scary. It’s not the information you are used to share (e.g. your name or email address), but all the information you wouldn’t consider sharing that makes this frightening. Big tech companies record data like income level, political and religious views, credit card information, your calendar events, all your search history and visited websites, and all the content you viewed or engaged with.

What information are giant tech companies collecting from you? Source: TruePeopleSearch

Uber stores massive amounts of data from its users, including their location, gender, spending history, contacts, phone battery level, whether they are on the way home from a one-night stand, and even if they are drunk or not. Uber has also experimented with its drivers to determine the most effective strategies for keeping them on the road as long as possible.

In order to place the right content in front of the right people, Netflix logs everything you have ever watched and how you watch: every time you click, watch, search, play, pause, what programs you consider watching but choose not to, and when you’re most likely to rerun a show. To better identify users’ preferences, content is categorized into tens of thousands of micro-genres, and then paired with a user’s viewing history. Everything you see on Netflix is a recommendation: the rows, the titles in those rows, and the order of those titles within the rows are all deeply considered.

But Amazon is a data powerhouse taken to a whole new level. They capture absolutely everything, from your product searches, what you look at but don’t buy, what you look at next, how you pay, how you prefer your shipping, your interactions with Alexa, or your requests to Echo. And the shocking thing is the level of detail they store: they capture what device you use, how many items you subsequently clicked on after selecting a product, your location, and the reading sessions and exact time of day for each tap on your Kindle device.

For Amazon, every mouse click and every twist and turn through its websites, apps and platforms are commodities that carry huge value.

When Amazon convinced third-parties to sell their items via their own marketplace, data collection skyrocketed and allowed the company to see way beyond their eye-sight: they can now access to any market they’ve ever wanted to, and see how customers behave in each one of them.

Tell me what I want

Each day you are influenced by algorithms that guide your decisions and choices. Algorithms are a step by step method for solving a problem or reaching a goal, based on taking an input and conducting a sequence of specified actions to reach a result. Since the explosion of modern technologies, they have expanded, sophisticated and replicated everywhere, having a central role in places like social media platforms.

The goal of several social media and content selection algorithms is to maximize clicks. They are designed to show or recommend stuff that will increase the probability of users clicking on it, since clicking is what generates revenue for the platforms.

For example, a click-through optimization algorithm is more profitable if it can better predict what people are going to click on, so it can feed them exactly that. So, a way to optimize the result is to feed users with content they like, and don’t show anything outside their comfort zone. Although it’s true that this causes their interests to become narrower, it’s not that the algorithms are trying to show you the stuff you like: they’re trying to turn you into a predictable clicker, taking you to a “predictable point” and making it easier for companies to perform any action (e.g. sell you something).

Companies have figured out that they can do this using your own data, by gradually modifying or emphasizing your preferences with targeted material. That’s basically, if you think of a spectrum of preferences, it’s to one side or the other because they want to drive you to an extreme, where you become a more predictable clicker and so they can monetize you more effectively. This is advanced applied behavioral science, or as Jeff Hammerbacher (the founder of Cloudera) said:

“The best minds of my generation are thinking about how to make people click ads. That sucks.”

The reasons behind this are mainly economic. A Wall Street Journal investigation found that Google manipulated search algorithms to prioritize large businesses over smaller ones, guiding search users to more prominent businesses over lesser-known ones. According to the investigation, eBay saw its traffic from Google nosedive during 2014, contributing to a $200 million decrease in its revenue guidance for the year. After Google told the company it had made a change that lowered the ranking of several high traffic pages, eBay considered pulling its roughly $30 million quarterly advertising spend from Google, but ultimately decided to start a pressure campaign on executives. Google eventually agreed to boost the ranking of several eBay pages that were demoted while eBay took on a costly revision of its web pages to make them more relevant.

Mechanized intervention is an ideal way to keep content flowing towards more lucrative topics, avoiding material that doesn’t generate engagement or profits. For tech companies to succeed, algorithms must focus on monetizable activities, and this is exactly what they are doing with our data.

Interested in these topics? Follow me on Linkedin or Twitter

from Featured Blog Posts - Data Science Central https://ift.tt/32MVDzQ
via Gabe's MusingsGabe's Musings