We aggregate and tag open source projects. We have collections of more than one million projects. Check out the projects section.
Uber, a taxi aggregation platform operates around the globe. It was started as monolith application and later re-architected as bunch of microservices, which gives them scalability. Uber uses lot of open source tools and they have contributed quite a lot projects back to the community. This article is about analyzing Uber's open source technology stack.
Programming Language:
Uber initially started with Python and later switched to NodeJS. They also use Java, Go, C++. Java and Go helps to build scalable, high performance and concurrent application.
Database:
Riak and Cassandra are used to meet high-availability, low-latency.
Schemaless, a inhouse product built on top of MySQL is used to store trips. Schemaless is an append-only sparse three dimensional persistent hash map, very similar to Google’s Bigtable. The smallest data entity in Schemaless is called a cell and is immutable; once written, it cannot be overwritten or deleted. The cell is a JSON blob referenced by a row key, a column name, and a reference key called ref key. The row key is a UUID while the column name is a string and the reference key is an integer. Schemaless is NOT open source.
JanusGraph - Scalable graph database optimized for storing and querying large graphs with billions of vertices and edges distributed across a multi-machine cluster.
LevelDB - Fast key-value storage library.
Caching:
Redis is used for caching
Twemproxy is used as a proxy for Redis, primarily to reduce the number of connections to the caching servers on the backend.
Load balancer, Service routing and discovery
When there are bunch of microservices, they should communicate with each other, they should be discoverable and provide high availability. A single API call, may internally call couple of other services. A co-ordination and high availability is required.
Nginx - Web server and also proxy's the request to backend servers.
HAProxy is used to provide high availability
TChannel - Network multiplexing and framing protocol for RPC and it has client in various programming languages.
Ringpop - Builds cooperation and co-ordination between application. It is resilient, client-agnostic sharding and fault tolerant
Thrift and Protobuf is used as Interface definition language (IDL) and helps in serializing data between RPC client and servers.
Development:
Phabricator powers a lot of internal operations, from code review to documentation to process automation.
Github is used for issue tracking and code review for open source projects
Jenkins does continuous integration
Puppet manages system configuration.
React to build user interface for data visualization
Radium set of tools to manage inline styles on React elements
Express.js - Bedrock webserver, is built on top of the this framework
D3.js is used for visualization
Mapbox - Interactive maps using mapbox
Picasso - A powerful image downloading and caching library for Android.
Gulp - Build system
Gradle - Build system for Android
Deployment
Production instances run Linux with Debian Jessie.
Docker containers on Mesos to run microservices
Apache Aurora is used for long-running services and cron jobs.
Clusto - Cluster management tool. It helps keep track of inventory of the infrastructure.
Logging
Logtron is used to log to disk and also push the events to Kafka.
Kafka is used to store events from various services.
ELK stack (ELK stands for Elasticsearch, Logstash and Kibana) is used to index and analyze logs.
Site reliability
Nagios aalerting for monitoring, tied to an alerting system for notifications.
Grafana - Metric analytics & dashboards
Apache Storm and Spark crunch data streams into useful business metrics.
References:
https://github.com/uber-common
Sponsored:
To find embedded technology information about MCU, IoT, AI etc Check out embedkari.com.
Subscribe to our newsletter.
We will send mail once in a week about latest updates on open source tools and technologies. subscribe our newsletterStackoverflow is one of most widely used Q&A platform. It is the default choice of developers. It has over 100 million page views in a month and its Google page rank is 60. Check out its open source technology stack, the tools used to build such a popular website. Stackoverflow uses open source projects and also contributes to it.
Magnolia CMS is one among popular java based CMS. It has support of CMS, DMS, Wiki, Forum and lot more features. This article discusses about the open source software used to build Magnolia CMS.
opentaps Open Source ERP + CRM, is a fully integrated application suite that supports eCommerce, Customer Relationship Management, Warehouse and Inventory Management, Supply Chain Management, and Financial Management to Business Intelligence and mobility integration out-of-the-box. This article discusses about the open source software used to build Opentaps ERP.
Facebook a leading social networking website predominantly uses open source technologies to build its application. Here is the list of open source products used and contributed by Facebook.
Twitter uses many open source products and also contributes most of the code to open source. Here is the list of open source products used by Twitter. This list does not include the projects sponsored by twitter.
WhatsApp is a most widely used cross-platform mobile messaging app for iPhone, BlackBerry, Android, Windows Phone and Nokia. Users can send text, images videos, audio for free. It almost replaced SMS. It changed the way people interact with each other. Lets check out the technology behind WhatsApp
Git is a fast, scalable, distributed revision control system originally written by Linus Torvalds. Git Server is initially written for Linux and later ported to Windows, Solaris and Mac. This article provides pointers to various Git server , Git clients, plugin and code review tools for Windows. It will be a Git complete solution stack for Windows.
Tumblr is a microblogging platform that allows users to effortlessly share anything. Tumblr now hosts over 70 million blogs with over 34 billion posts to date. Below is the list of open source used in Tumbler.
LinkedIn is a social network for professionals. LinkedIn handles millions of searches as well as hundreds of thousands of updates daily. They sponsored many projects to open source. Here are the list of open source products used by LinkedIn.
Lily currently offers an open source content repository. It is the first cloud-scalable repository for social content applications. It is built from ground up using Big Data and NOSQL technology. Its technology stack includes Hadoop, HBase and Solr.
Zimbra is a Enterprise messaging and collaboration software. It is a good alternative to Microsoft Exchange server. Zimbra is a Email Server but they have not written anything related to SMTP server, rather they have integrated well know open source email server in to its package. Zimbra is a system of well connected / integrated multiple open source software which delivers enterprise quality to it. This article explains most important components which helps to build the enterprise product.
Email is the primary way of communication. Small organization which hosts their website / any other application from a third party hosting service providers may require this email stack.
SourceForge.NET is most popular and widely used Forge. It helps to host software projects. It has integrated support for Wiki, Forum, Tracker and Full text search. The code base named allura, is completely built on open source stack. This article explains few important one used to build SourceForge.NET.
Email is most important in any organization. Lot of business are happening via Email. This article discusses about the end to end complete Email solution for an organization. What an organization requires? Email spam filtering, Email Server, Email Archiving and Email client. If all are open source then complete solution could be achieved free of cost.
Pinterest is a tool for collecting and organizing things you love. It is a social networking site where users could pin images and write a note for that. It is now currently serving billions of pages every month. Check out the open source products used in Pinterest.
OpenJDK is the open source implementation of Java Standard Edition. Oracle (Sun) JDK implementation is going to be based on OpenJDK. OpenJDK and Oracle JDK are almost similar. There are ready made binaries available for Linux distribution but you may need to build OpenJDK on windows.
Subversion version control system is most popular and widely used. Windows users always want easy to install, easy to configure and easy to use applications. This article explains about few set of tools which used in combination will bring you complete Subversion solution in windows.
Barack Obama election fundraising campaign was hosted online and the platform was built using open source products. The campaign stayed for 6 months and it has more than 8 million page views. This article explains the open source products used to build the platform.
Wikipedia is a multilingual, collaboratively edited encyclopedia. It is one of the busiest site in the world. It has more than 8 million articles and accessed by millions of users around the world. This article briefly discuss about the open source software used in Wikipedia.
Meta Search engine is nothing but a search engine which searches more than one search engine and combines or filters the results. Each search engine has its own proprietary ranking mechanism to rank the results. When combined the search results from all leading search engines would be more informative and useful. With less page traversals we will end up our destination.
We have large collection of open source products. Follow the tags from
Tag Cloud >>
Open source products are scattered around the web. Please provide information
about the open source projects you own / you use.
Add Projects.