Tendrl – The Unified Storage

Tendrl – The Unified Storage

Hey, I started working on an upstream project, Tendrl, 6-7 months back. Tendrl is also called as Red Hat Gluster Console Web Admin (RHGS-WA) as downstream.

Tendrl is Open Source project designed to manage today’s and tomorrow’s SDS ( Software-defined Storage ) technologies. The user interface is based on open interface project, Patternfly.

Tendrl provides installation, storage provisioning, lifecycle management, monitoring, alerting, trending, and analytics for Gluster storage, while providing a framework to encompass other SDS technologies in the future. So in short, its a project for developer by developers.

I am working as a UI developer. The project is based on AngularJS 1.5x.(soon will be moving to higher version for Angular). This blog is about the current structure of Tendrl UI (as to how data is flowing) and a quick guide to set up a Tendrl setup from scratch.

Structure of UI

The table briefly explains the structure of current UI of Tendrl.

1. docs : Contains the guidelines to develop or deploy Tendrl on your systems.

Note: The updated one is present here.

2. src : It is the place where magic begins, all the source and configuration is here. The data that comes here, is processed to be shown in various views. Also, Tendrl UI is divided into different views under modules, which makes the code clean and understandable.

Flow of data is carried out in following manner:-

  • Api: This folder contains mock data of api’s. This mock data is useful for testing the app.
  • commons: It is just like a manufacturing place, when raw material is added and processed. It receives and sends the data to and fro from backend.
  • Images: It contains the images required for application.
  • Modules: Every important part of Tendrl is divided into various views, and each view has its own controller and css rules. So every view in Tendrl is specified as a specific module. The sub-views are there as sub-folders.
  • config.json: It contains the configuration settings for entire app. NOTE: The setup to be prepared has to be placed "baseUrl": "IP of VM".
  • index.html: The place from where the application starts.

3. test : There are many test that are performed on UI, in order to check the UI representations, to see if they are consistent with the expected ones, the input and expected values are stored as the mock data in api folder. Tendrl now just have unit test for few modules ( WIP ). We can also write Behavior Driven Test or Behavior Driven Development (BDD) for the application.

4. .gitignore: Specifies intentionally untracked files that git should ignore.

5. .travis.yml: Verify code on github. You can see whether builds are passing, test are failing, who did what. It is triggered whenever code in repository changes.

6. Makefile: These are special format files that help build and manage the projects automatically.

7. README : This markdown file explains how one can contribute to Tendrl UI.

8. gulpfile.js: Task Runner file for Tendrl.

9. package.json:  This file holds various metadata relevant to the project. This file is used to give information to npm that allows it to identify the project as well as handle the project’s dependencies.

10. tendrl-ui.spec: specifies Tendrl release change set.

How to setup Tendrl ?

Note:- these will help you setup the server with minimum packages
1. Make 3 VMS of centos OS – two will act as gluster-servers and one as tendrl-server, you can make as many gluster-servers you want (I am making two for simplicity).

Let Tendrl-server be x.x.x.x and gluster-server nodes be y.y.y.y1 and y.y.y.y2.

2. Login into VMs.

Note: VMs should have a network interface ie. they must have internet connection.

3. Change hostname in terms of fqdn ie. for all the VMs, execute following:-

$ ip addr

Then, note down the IP address ( according to screenshot the IP adress is 192.168.122.158) . Change the following file: /etc/hostname – Replace the hostname in terms of IP address, for example if the ip address is 192.168.122.158 and /etc/hostname has localhost.localdomain, it can be replaced by dhcp122-158.lab.com.

Save the file ( Press ESC and type – :wq )

4. Now, update /etc/hosts file on all VMs like following:-

ie. add the ip address of the vm followed by the hostname you provided.

5. ‎The server node should communicate with other nodes using ssh keys. So ssh-keygen on server node and and ssh-copy-id it on various gluster nodes.

[ root@x.x~] $ ssh-keygen -t rsa -b 4096 -C "your_email@example.com"
Note: make a password less ssh-key
[root@x.x~] $ ssh-copy-id root@(hostname)

ssh key should be copied to all VMs, including the server itself.

To verify try doing ssh using hostname.
[root@x.x~] $ ssh root@(hostname)

Now follow the below steps on sever node only (x.x.x.x)
1. Install following packages:-
yum install python-dns git ansible

2. ‎Clone tendrl-ansible repo -> https://github.com/Tendrl/tendrl-ansible/
[ root@x.x~] $ git clone https://github.com/Tendrl/tendrl-ansible.git

3. Following steps for installation or you can also follow README file:-
[root@x.x~] $ cd tendrl-ansible

* Make an inventory file
[ root@x.x~] $ vi inventory

* Enter in the following format:-
[gluster-servers]
Hostname of y1
Hostname of y2

[tendrl-server]
Hostname of x

Note: Hostname can be know by running hostname in terminal.

* Create site.yml from site.yml.sample
[root@x.x~] $ cp site.yml.sample site.yml

* Small hack for running tendrl on local vms.
In site.yml change etcd_ip_address: "{{ lookup('dig', ansible_fqdn) }}" to etcd_ip_address: "0.0.0.0"

* Validate Ansible by :

[root@x.x~] $ ansible -i inventory -m ping all

* If everything looks fine run following command:

[root@x.x~] $ ansible-playbook -i inventory site.yml

Now your setup is ready, put the IP address of server machine on your browser, and you will see the following:-

 

The default username and password is admin and adminuser.

For development, you need to place the ip address of the server created in config.son as "baseUrl": "http://192.168.122.8/api/1.0/", then you can follow the README file for Tendrl UI.

Following errors are expected:-
1. Ansible_memtotal : Go to precheck.yml and change the configuration to suitable. Like following:-

Edit the file according to your configuration:-


2. Version of centos <= 7.3 : Upgrade your os,
[root@x.x~] $ yum upgrade

3. NTP not configured: NTP has to be synchronized.

[root@x.x~] $ yum install ntp
[root@x.x~] $ vi /etc/sysconfig/ntpd – replace OPTIONS with following
OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid"
SYNC_HWCLOCK="yes"

[root@x.x~] $ systemctl start ntpd
[root@x.x~] $ systemctl enable ntpd

4. No Unique machine-id
Check weather all VMs have same machine id by $ cat /etc/machine-id
If they are same, run following commands on any two machines
[root@x.x~] $ rm /etc/machine-id
[root@x.x~] $ systemd-machine-id-setup

You can also go through: Tendrl Install guide.

Stuck somewhere? Please feel free to comment or reach me.

Advertisements

Google Developer Days’17, India

Google Developer Days’17, India

I got a chance to attend Google Developer days that took place on 1st and 2nd December at Bengaluru International Exhibition Centre. It was an amazing experience. This blog is about my experience for these two days.

People from various parts of world gathered to be a part of it. I traveled to conference with Devyani Kota and Sayan Chowdhury. We reached at around 10:20 am at Nagasandra metro station, we were received by GDD volunteers and we headed towards BIEC. The BIEC was decorated with colors of Google.

Since it was house full for every session, I got to attended only 3 sessions in person and 2 virtual sessions, following are some points or keywords that interest me about the session:-

1. Introducing Cloud Firestore by Dan McGrath (Product Manager, Google Cloud)

  • Talk was about Google newest Database, Cloud Firestore
  • It was inspired by Firebase real-time database.
  • It is a NoSQL ducument-oriented database which can scale automatically and query data with multiple properties.
  • It also uses JavaScript, which is nice.

2. Containers, Kubernetes, and Google Cloud by KC Ayyagari (Program Manager at Google)

  • The talk was about Google Kubernetes Engine which is a sort of manager service for Kubernetes.
  • He highlighted the difference between physical machines, virtual machines and containers.
  • Importance of container orchestration engine – Kubernetes.

3. Quality over Quantity: How to Use Material Design and Android Vitals by Johannes Tonollo (UX designer, Google Play), Nick Fortescue (Software Engineer, Google Play)

This session was short and clear.

Explaining why/what ‘quality’ means in terms of App on Google play store. I never knew there are so many ‘app vitals’ that an app on Google play store is measured upon, like analyzing stability through Application Not Responding(ANR) rate, crash rate, analyzing battery of device by rendering and frozen frames, A/B testing and various things. They discussed about the Google’s product – Android Vitals to test the performance of the app by using various real time monitoring, great filters, benchmarking values, pre-launch reports and many more.

4. Mobile in Context: Design Principles of Flow and Navigation by Jenny Gove (UX researcher)

It was quite an interesting session. She explained Googles current Android (Material) design from the perspective of an UX designer. She mentioned about the constant increase and decrease in the demand of web apps over native apps due to design and ease of use. Google has done a great job to improve the experience of web. She focused on elements of flow like navigation and explained what/why/how are different designs of navigation used based on the app concept and user behavior.

5. Design Thinking Workshop by University Innovation Fellows

It was a design workshop where we were divided in a group of 7-8 each and assigned a mentor. Every mentor is a university fellow of Stanford d.school. As a group we discussed thoughts and ideas to solve a problem. It was a nice way to analyze different types of thought for a common problem. It was a good interactive session.

I got this after this session:-

In all, the event was nice, the food was good :p , the props and goodies were exciting, the speakers were great and talks were google product centric, but knowledgeable. I got to meet lots of like minded people, also got to meet some friends from Delhi 🙂 and made new friends. The thing attracted me the most is creative work done by Google design team for this event and the products that were highlighted. I would be looking forward to Google Developer Days’18 for sure.

Fedora Women Day 2017, Bangalore

Fedora Women Day 2017, Bangalore

Better late, than never.

Fedora Women’s Day’17 was celebrated in India on 16th September. The event took place in one of the esteemed university of Bangalore, Christ University. I got a chance to be a speaker for the event. It was organized by Fedora Ambassador and current Fedora Diversity head Amita Sharma. She works as Senior Quality Engineer at Red Hat, Pune. Other organizers were Sumantro Mukherjee, Fedora Ambassador and Associate Quality Engineer at Red Hat, Bangalore, Vipul Siddharth and Prakash Mishra, who are Fedora QA intern at Red Hat, Bangalore and students at Christ University.

The event started at around 12 pm, quite a no. of students turned up for the event and most of them were girls, which felt good :). Event started with Vipul introducing all the speakers.

Amita talked about women in Fedora and how can one start contributing. She also highlighted the problems faced by women around world and how can they overcome that by technology.

As fedora design is a team where most of the contributors are women, I decided to talk about it. I talked about various initiatives under Fedora design and how can one start with it.

Next talk was given by Sarah Masud, she works as Data scientist in Devtools team of Red Hat, Bangalore. She is one of the women who has achieved a lot in such a young age. She talked about Python Pandas, one of python library providing high-performance, easy-to-use data structures and data analysis tools. It was quite an interesting talk.

The event ended with lot of interaction between students, speakers and teachers.

I would like to thank all the organizers, the Fedora project and Labyrinth club of Christ University for all the goodies 🙂 and making this event possible.

Inspired by blog by Amita Sharma.

 

Flock 2017 : The Fedora Conference

Flock 2017 : The Fedora Conference

Its really awesome to travel to meet people who inspired you and whom you can only meet online.

On 29th August, I got a chance to fly down to Hyannis, Massachusetts to attend Fedora global Conference, Flock’2017. Being contributing to Fedora for more than an year now, it was my second Fedora conference other than local meetups and Fedora release parties. Also, it was my first trip to US :).

I was working on a Symbol library under the guidance of Mairin Duffy, my next blog will talk more about this library.

I was traveling with Sumantro Mukherjee, who is newly elected member of Fedora Ambassador Steering Committee and Fedora Qa contributor 🙂 and Sinny Kumari, who is contributing to Fedora architecture for long now. We boarded our flight from Bangalore Airport on 29th August morning, although we were little worried about jet lag yet we were full of excitement and enthusiasm.

We landed at Logan International Airport of Boston at around 4 pm and took a bus to Hyannis. We reached the venue (The Resort and Conference Center at Hyannis) late around 9pm. The room was clean and pretty. The view from my room was a never ending golf course which seemed amazing.

Conference day #1

Flock started with a powerful opening speech given by Brian Exelbierd and Matthew Miller. Also, I volunteered for Fedora Badge Design Workshop where I explained about my project, how can they use it, its advantages and disadvantages, you can find my slides here. The workshop was organized by Marie Nordin and Masha Leonova, I managed to make one badge in the workshop -> https://pagure.io/Fedora-Badges/issue/342. After that I headed towards Fedora Magazine session taken up by Paul Frields, Ryan Lerch and Justin W Flory, it was quite interactive session, I enjoyed the discussion and also gave suggestions.

Conference day #2

It was raining, so we had breakfast in the resort itself, it was good and not too expensive. After that I attended a session on Simple User Testing- early and often by Jennifer Kotler, although I was little late but I managed to gather few important points at the end. She explained how can one write unit tests for particular type of mockups, it was great the way she explained with examples. The next session I attended that day was Fedora Hubs Hackerfest, since I have contributed to this project in past, so I really wanted to attend this session. It was taken by Aurelien Bompard and Sayan Chowdhury. They made widget making in Hubs so easy :).

Conference day #3

I went out for breakfast with Sumantro Mukherjee and Sayan Chowdhury, walking on streets of Hyannis was an another experience altogether, the roads were peaceful and beautiful. We clicked a lot of photos on the way.

This day I attended Suzanne Hillman‘s session on her UX design case study for Fedora contributors as part of her Outreachy project, I really liked this session as she explained everything she has been doing in her project. The next session was on Fedora design pattern library given by Mairin Duffy. She introduced us the project PatternLab that she is working on and how we can contribute to it. This session was followed by a Design Hackerfest where we started contributing to PatternLab project.

Later on this day, I went out for dinner with Adam Williamson, Lukas Brabec, Tim Flink and Sumantro Mukherjee. I was really glad to meet them as a team member.

Conference day #4

The last day was all about thank giving and winding up session. I met all the people I could meet.

It was great to meet and talk to Adam Willamson and Mairin Duffy, as I have talked to them many times over mails and IRC. Also, I got to meet Robert Mayr (robyduck), Sachin S. Kamath, Justin W Flory, and many more great contributors.

All in all, It was a great experience and I would like to thank Fedora community for inviting me for this conference.

I also explored Boston and New York City on this trip. Glimpse of my trip:-

 

 

 

Technologix’17 : SJCE, Mysore

Technologix’17 : SJCE, Mysore

On 16th March’17, Sri Jayachamarajendra College of Engineering (SJCE), Mysore invited Red Hat to be the part of there ongoing CSI fest, Technologix. Red Hat was the premium sponsor of the fest.

The event was a success with fresh energy, most of the students were sophomors and were aware about Fedora, Linux and opensource.

Four speakers, including me, decided to go and talk about different technologies which are being used at Red Hat extensively. We travelled from Bangalore to Mysore in a car, with made the journey more exciting. It took us nearly 3.5 hours to reach our destination.

On reaching there, Vishnupriya, a student from SJCE, welcomed us and guided us to the guest rooms.

The session started at 2:30 pm with about 150-200 students waiting to learn about technologies used in industry. The CSI members welcomed us with flower bouquet and inspiring words.

Virtualization

The first talk was given by Buvenash Kumar, who is engineering intern at Red Hat. He has been working with Red Hat for last 1 year. He talked about virtualization, different types and benefits of virtualization. He also talked about various hypervisors in market like Red Hat Enterprise virtualization, VMWare Esxi, Xenserver.

Ansible

The talk on Ansible was given by me. I talked about SSH, Introduction to Ansible, Introduction to YAML, Setting up test environment, Ansible config and HOST files(demo).

Openstack

The talk was given by Ankit Raj. He was a part of GSOC’2015, and contributed to Ruby community. He works on gluster project as Associate software engineer at Red Hat. He talked about the Openstack and it’s usage, OS searchlight concepts and flow and its contribution guide.

Git and contribution to opensource

This session was taken up by Sumantro Mukherjee who is Associate Quality Engineer, Fedora Project, Red Hat and Arvind Chembarpu, who interns as Fedora QA at Red Hat.

The event ended on a great note and a small speech by Dr. SK Padma, the head of department of computer science SJCE, she mentioned about the opensource wave in there campus and how they promote opensource. The speakers were awarded trophies.

After the event, we explored the beautiful campus of the college.

Special thanks

CSI chapter of SJCE, Sudhir Dharanendraiah, alumni of SJCE and QE Manager at Red Hat, Red Hat organization and Arvind Chembarpu for making the visit possible. And thanks Rejy for dvds.

Fedora Activity Day, Bangalore 2017

Fedora Activity Day, Bangalore 2017

The Fedora Activity Day (FAD) is a regional event (either one-day or a multi-day) that allows Fedora contributors to gather together in order to work on specific tasks related to the Fedora Project.

On February 25th ’17, FAD was conducted in one of the admirable university of Bangalore, University Visvesvaraya College of Engineering(UVCE). It was not a typical “hackathon” or “DocSprint” but a series of productive and interactive sessions on different tools.

The goal of this FAD was to make students aware about Fedora so that they can test, develop and contribute. The event was a one-day-event, started at 10:30 in morning and concluded at 3 in evening.

The first talk was on Ansible, which is an IT automation tool. It can configure systems, deploy software, and orchestrate more advanced IT tasks such as continuous deployments or zero downtime rolling updates. The session was taken up by Vipul Siddharth and Prakash Mishra, who are Fedora contributors. They discussed about the importance of such automation tool and gave a small demo for getting started with Ansible.

The Ansible session was followed by the session on contributing to Linux Kernel, given by our esteemed guest Vaishali Thakkar (@kernel_girl ). Vaishali is Linux Kernel developer at Oracle, she is working in kernel security engineering group and associated with the open source internship programs and some community groups.Vaishali highlighted upon each and every aspect of kernel one should know before contributing. She discussed the lifecycle and how-where-when of a pull request. The session was a 3 hour long session with a short lunch break. The first part of the session was focused on theoretical aspects of sending your first patch to kernel community and the second part was a demo session where she sent a patch from scratch (Slides).

The last session was taken up by Sumantro Mukherjee (Fedora Ambassador) and me, on pathways to contribute to Fedora with a short interactive session.

The speakers were awarded tshirts as a mark of respect.I would like to thank Sumantro Mukherjee, Fedora Community and IEEE subchapter of UVCE college for making FAD possible.

Fedorahosted to Pagure

Fedorahosted to Pagure

Fedorahosted.org was established in late 2007 using Trac for issues and wiki pages, Fedora Account System groups for access control and source uploads, and offering a variety of Source Control Management tools (git, svn, hg, bzr). With the rise of new workflows and source repositories, fedorahosted.org has ceased to grow, adding just one new project this year and a handful the year before.

As we all know, Fedorahosted is shutting down end of this month, its time to migrate your projects from fedorahosted to one of the following:-

  1. Pagure
  2. Hosting and managing own Trac instance on OpenShift
  3. JIRA
  4. Phabricator
  5. GitHub
  6. Taiga

Pagure is the brainchild of Pierre-Yves Chibon, a member of the Fedora Engineering team. We will be looking into Pagure migration because Pagure is a new, full featured git repository service and its open-source and we ❤ opensource.

So, Pagure provides us Pagure test instance where we can create projects and test importing data. Note:from time to time it is been cleared out, so do not use it for any long-term use.

Here is How Pagure will support Fedorahosted projects ?

Features Fedorahosted Pagure
Priorities We can add as many priority levels as required with weights Same
We can assign a Default priority No such option
Custom priority tags Same
Milestone Ability to add as many milestone as we want Same
Option to add a due date Same
Keeps a track of completed time Does not record completed time
Option to select default milestone No such option
Resolution Ability to add as many resolutions as we want Same
Can set a default resolution type By default it is closed as ‘None’
Other Things Have separate column for Severity, component, Version Here it is easy, it has just Tags
Navigation and Searching Difficult Easy
Permission Different types of permission exist Only, ‘admin’ permission exist
Creating and maintaining tickets Difficult Easy
Enabling Plug-ins Easy Easy

So, lets try importing something to staging pagure repo, I will be showing demo using Fedora QA repo, which has recently been shifted from fedorahosted to pagure.

  1. You should have XML_RPC permission or admin rights for fedorahosted repo.
  2. We will use Pagure-importer to do migration.
  3. Install it using pip . python3 -m pip install pagure_importer
  4. Create a new repo ex- Test-fedoraqascreenshot-from-2017-02-10-16-53-21
  5. Go to Settings and make the repo, tickets friendly by adding new milestones and priorities.screenshot-from-2017-02-10-16-56-50
  6. Clone the issue tracker for issues from pagure. Use: pgimport clone ssh://git@stg.pagure.io/tickets/Test-fedoraqa.git.This will clone the pagure foobar repository into the default set /tmp directory as /tmp/Test-fedoraqa.gitscreenshot-from-2017-02-10-18-28-20
  7. Activate the pagure tickets hook from project settings. This is necessary step to also get pagure database updated for tickets repository changes.screenshot-from-2017-02-10-18-30-19
  8. Deactivate the pagure Fedmsg hook from project settings. This will avoid the issues import to spam the fedmsg bus. The Hook can be reactivated once the import has completed.
  9. The fedorahosted command can be used to import issues from a fedorahosted project to pagure.
    $ pgimport fedorahosted --help
        Usage: pgimport fedorahosted [OPTIONS] PROJECT_URL
    
        Options:
        --tags  Import pagure tags as well.
        --private By default make all issues private.
        --username TEXT FAS username
        --password TEXT FAS password
        --offset INTEGER Number of issue in pagure before import
        --help  Show this message and exit.
        --nopush Do not push the result of pagure-importer back
    
    
    $ pgimport fedorahosted https://fedorahosted.org/fedora-qa --tags

    This command will import all the tickets information with all tags to /tmp/foobar.git repository. If you are getting this error: ERROR: Error in response: {u'message': u'XML_RPC privileges are required to perform this operation', u'code': 403, u'name': u'JSONRPCError'}, means you dont have XML_RPC permission.

  10. You will be prompted for FAS username and password.screenshot-from-2017-02-10-19-01-51
  11. Go to tmp folder cd /tmp/
  12. Now, we need to push the tickets to new repo. The push command can be used to push a clone pagure ticket repo back to pagure
    $ pgimport push Test-fedoraqa.gitscreenshot-from-2017-02-10-19-10-04 screenshot-from-2017-02-10-19-10-16
  13. Refresh your repo, and it will look like thisscreenshot
  14. Now you can edit tickets in any way you want.

Stuck somewhere? Feel free to comment and contact. Thanks for reading this 🙂