| Metadata | |
|---|---|
| cEP | 0032 |
| Version | 1.0 |
| Title | Enhance Websites Architecture |
| Authors | Keshav Garg mailto:co16326.ccet@gmail.com |
| Status | Proposed |
| Type | Feature |
This document describes the features that will increase the usability and easiness for all the open-source enthusiasts. And, that will be accomplished by enhancing and improving the coala Community and Projects website for newcomers, developers as well as for other communities. Also, new API Endpoints will be implemented in coala Webservices where many of them will have their respective cron jobs. CRON Jobs are necessary because it will help us to keep the database up-to-date by regularly fetching the members contributions to the organization, Netlify form submissions, and the activities happening within the organization like new issues opened, etc. These new endpoints will manage the organization database in a much more structured form.
A website always plays an essential role for an open-source community as it spreads the idea of offerings that community provide to other organizations, with information about its active community by telling about their skills as well as about their precious contributions to the organization. And to provide such offerings this project aims to provide useful features to contributors. For contributors, who haven't been part of the organization can fill-up the Community Join Form. The form submission will checked by performing some checks, to validate whether you're eligible to be a member of the organization or not. And for the organization members, some new below-mentioned features will be implemented so as to make the tasks easier within the organization, for example, getting assigned to an issue via a form. Information about all the new features, are being explained in further sections.
coala Community website
The listed enhancements will ease the navigability and usability for newcomers, developers and to all others who wanted to be a part of the organization. Implementing these enhancements will give community members a glance about the precious contributions made by you as well other contributors to the organization. And, you can even view your contributions in the form of charts. Following are the new enhancements that will be added to coala community website:
-
Display a web-page for -
-
organization teams
This web-page will display information about the organization teams, and will be displayed to only authenticated users. It will definitely not show all the contributor's name, as we already have a web-page which displays the all the members of the organization. So, what will it show? It will definitely not show what permissions are available to team members. But, it will definitely provide information like What does this team aim to do?, Members count, and a count value which gives information about 'how many new members are being added to this team this week?'. Clicking on any team name will redirect the user to either on GitHub showing all team members or a new web-page which will show all the team members of this team.
-
contributors' geolocation in the form of a map
Showing the global influence of an organization around the world of open-source organization's can be achieved by showing the contributors' geolocation. It can be shown in the form of a world map which displays the locations in the form of a cluster. Each cluster demonstrates a number of contributor's around the world, who have contributed to the organization in any manner. Only the organization members geolocation will be shown-up on the map, for now.
-
mentors
It will be good if community website can show-up all the potential mentors that an organization has. It will allow other organizations to get to know more about them.
-
-
Add a Netlify form for -
-
adding a new calendar event,
-
adding new Open-Source Google forms,
-
adding details about your Google Summer of Code(GSoC) project,
-
applying as a mentor in GSoC,
-
joining the community (newcomer form),
-
getting promoted to developer role(Promotion to Developer form), and
-
Feedback
Feedback from organization developers or users is necessary for every organization, therefore an addition of feedback form to community website will be very helpful.
Following Validation checks will be performed while:
- adding a contributor as a mentor
- Participated in Google Summer of Code with the organization, and
- is a member of the organization
- adding yourself as a GSoC participant
- cEP(coala Enhancement Proposal) Merged
- Project details are valid, for example -
- At-least one accepting Review from any of the maintainers
- Issue that gives idea to solve project proposal has a label of “Project/Proposal”
- Valid project link to projects website
- Only organization developers or contributors who are members of more than one team will be able to add new open-source google forms details as well as add new calendar events details.
- Joining the coala community (for newcomers)
- Already not a member of org!
- Hasn't joined GitHub/GitLab recently (i.e. not created account in last 1 hour)
- has pushed at-least 1 source repositories on GitHub/GitLab
- has done any of the github training exercises etc.
-
-
Login/Signup using GitHub & GitLab
The authentication will be a way to avoid spam activities or to avoid any unwanted update by the newcomers in the database. Also, we can’t use ORG API Token for updating user related information as there will more tasks which have to completed first. So, using user authentication we can get user their Access Token and use it to update their contributions related information.
Adding auth will allow us to make sure that newcomer can’t:
- Upload any google form
- Participate as a mentor
- Update the database
- Apply for promotion to developer for any other user
- Assign someone else to the issue
-
Manage community members data
We can't use all the time organization access token to manage the contributors live data. Using an access token created by user by logging-in, will allow us to manage their data. Using the access token, the user contributions to the organization can be maintained in the database. Every time a user log-in, the access token will be refreshed in the database.
-
Gamification & Meta-Review rank leader-board on homepage
Each table will show top 5 contributor's based on there contributions, they've made in the organization.
-
Contributor Statistics
Since, we'll get user access token when they log-in on the community website. Using that, we will be fetching the contributions in a detailed form and store them in the database. This data will be used to display the contributor statistics in the form of charts. The statistics pop-up display, will have a nice UI/UX to make it more interactive. It will display the charts in 2D, that shows hike in contributions, activities, overview of contributions and many more. Along with, the issues on which user has worked-on will be displayed (just labels).
-
Improve Gamification leader-board UI/UX design with added filters
The current layout uses just HTMl w/o any UI/UX. The improvement will add an interactive UI/UX design to it. Along with adding design, filters option and a search bar will be provided. The filters can be used to filter participants based on the score range, gamification level and the badges.
-
Automate assigning of issue to contributors
A netlify form will be created with relevant input fields which will be helpful in assigning an issue to the contributor.
Based on following checks, the issue will be assigned to avoid spam-
- Not assigned to any other issue
- If assigned, the issue has a PR to close it
Since corobo does this work for us of assigning an issue to the contributor, but it has been down from a long time and no idea when it will up again for helping contributors. corobo provides features like invitation to newcomers, creating an issue, assigning an issue and many more. But due to maintenance it is down. So, this feature on community website can be treated for safe side. As well as it will help contributors by not spamming on coala main gitter channel for getting assigned on issue. Let's just keep that channel for coala related talk.
According to study published by adobe, 38% of people will stop engaging with a website if the content/layout is unattractive. Therefore, this section describes some major improvements in coala community website. Following are the improvements:
-
UI/UX Design improvement
- Add materialize.css
- Re-design all webpages
- Gamification leaderboard
-
Update README.md providing more useful information
-
Updated Header image
The new header image will symbolise the meaning of creating, supporting and growing community.
-
Why we need this website? (Role of community)
-
Describing features of coala community website
-
-
Update coala newcomer docs about automation features
coala Webservices website
-
New database models with endpoints
To enhance the website architecture these new models will play a vital role. It will manage all the required database records that will shown on coala community and projects website with which the user will be interacting. Following new models will be added to coala webservice api-
- Calendar
- Google forms
- GSoC
- Mentors
- Feedback
In addition to above new models to be created, the existing models will be improved to manage more database of the organization and contributor. The existing models that will be improved are-
- Organization Teams
- Contributor
-
Keep contributors' geolocation cluster map up-to-date
To keep up-to-date contributors' geolocation, a new cron job will be added to coala webservices that will refresh the cluster map with updated geolocation of users. Then this map will be included on the Community website to show the global influence if the organization.
-
Website homepage to display information about new enhancements
The information related to new endpoints created to enhance the websites architecture like endpoint name, url and what it does? will be added to coala Webservices homepage.
-
Add Continuous Deployment to Webservices
Fix existing problems
The existing website is having some flaws which needs to be fixed-
- There are some error's with existing cron jobs like the GitLab related cron's are running efficiently and even the data isn't being displayed.
- coala Webservices is not being used by coala websites to fetch organization and contributors related data which is being displayed to users on both the websites. Instead it is getting static data from @sks444 RawGit repository.
coala Projects website
-
Advanced filters for searching projects
It should be easy for a user to search among the GSoC projects according to the requirements like-
- Difficulty level,
- Initiatives,
- Project tags,
- Project Status, and
- Collaborating Projects.
-
A tab for google forms
Thanks to @RohanVB for proposing this idea. Since most of the forms are posted on gitter and get lost with messages within a day, so why not to have a tab with an archive of all of the important forms? It will be one- stop for contributors to get all the important forms instead of just scrolling up on the gitter in search of the forms.
-
A calendar web page showing important events and related info.
The calendar will basically be used to show GSoC (and most probably GCI) timeline, with milestones events created by coala GSoC'ers. Also, an ability for project members to publish periods of extended unavailability, if they want to, before and during GSoC. This can be done from coala community website from their profile where they can do this.
Also calendar will be used to record significant project related events / posts, such as software releases, survey result announcements, cEP merges which affect everyone (e.g. GSoC responsibilities changes, or CoC updates).
-
A tab for GSoC students
From past 5 years coala have been a part of GSoC and GCI like programs. But a disappointed fact that GSoC'ers might have in the mind that the organization doesn't publish any news about them. The organization must somehow display the selected contributors for GSoC and GCI current year program as well as for past years. It allows newcomers too to get motivated. This tab will -
- Show all students that have been selected for GSoC in past years.
- Design - A Collage containing all GSoCers for a year or
- just display like admins/mentors are being shown under mentors tab
- Re-design homepage
- Add UI/UX design to GSoC project reports page
- Responsive UI of a GSoC report page (eg. Project Status Report)
- Mark issues state in project description popup gsoc The background color of related issues mentioned in project description popup should be decided on their current state.
- It is important for any organization to show its global influence by showing how people from different countries come together.
- They join open source projects and make that project more usable or great so that it can aid other developers as well as communities.
- To stay updated, the map will be regularly updated using a cron job.
- The following code will be used to create a cluster map to show coala members geolocation.
from github import Github
import getorg
gh = Github(login_or_token=<ORG_TOKEN>)
map, org_location_dict, org_metadata_dict = getorg.orgmap.map_orgs(gh,"coala")
getorg.orgmap.output_html_cluster_map(org_location_dict,"cluster_map")- As a result, a html file will be created showing a cluster map which we can embed in any of the HTML pages.
After the models have been created, we can connect websites to new API endpoints and their backend to add some data which can be further used to display on websites.
urlpatterns = [
path('^forms/', AddGoogleForms.as_view(), name='Add Google Form'),
path('^feedback/', AddFeedback.as_view(), name='Add Feedback'),
path('^gsoc/mentors/', AddMentor.as_view(), name='Add Mentors'),
path('^gsoc/participant', AddGSoCParticipant.as_view(), name='Add new participant'),
path('^users/<str:USERNAME>', UserProfile.as_view(), name='User profile'),
]Each endpoint will be have its own backend functionality to add data to the models database. And, for issues related data I will be creating a cron Job. A sample code snippet for GSoC Participant and Google Forms
-
GSoC
-
A database model to have information about all past GSoC students.
-
The contributor can add information to database from their profile section after or during GSoC.
-
The POST method will be having rudimentary checks to avoid spamming.
-
On API endpoint following view(in views.py) will be called:
from .serializers import GSoCParticipantSerializer from .models import GSOCParticipant class AddGSoCParticipant(ListCreateAPIView): permission_classes = [] authentication_classes = [] serializer_class = GSoCParticipantSerializer queryset = GSOCParticipant.objects.all() # POST method for adding GSoC students def post(self, request, *args, **kwargs): return self.create(request, *args, **kwargs)
-
For handling/validating the POST data, a serializer class will be created as follows:
from .models import GSOCParticipant Class GSoCParticipantSerializer(serializers.ModelSerializer): class Meta: model = GSOCParticipant fields = ('name', 'topic', 'year', 'personal_image') def validate(self, data): # Perform Validation on POST data return data
-
-
Google Form
-
New google forms can uploaded via profile by only developers not by newcomers.
-
GET Method will be called on projects website & POST Method on community website
-
Handling Google Form view (views.py)
from rest_framework.generics import ListCreateAPIView from .serializers import GoogleFormSerializer from .models import GoogleForm class AddGoogleForms(ListCreateAPIView): permission_classes = [] #TODO Newcomers can’t add forms authentication_classes = [] serializer_class = GoogleFormSerializer queryset = GoogleForm.objects.all() # POST method for adding new forms def post(self, request, *args, **kwargs): return self.create(request, *args, **kwargs)
-
Google form respective serializer
from rest_framework import serializers from .models import GoogleForm class GoogleFormSerializer(serializers.ModelSerializer): class Meta: model = GoogleFormSerializer fields = ('title', 'url', 'description', 'end_date') def validate(self, data): # Perform validation of POST data return data
For other API endpoints mentioned above, similarly backend functionality will be added with appropriate functions/classes defined in views.py and serializers.py.
-
-
It should be easy for newcomers to join and contribute to an open-source organization.
-
It might happen that organization maintainers are not active which may lead to late invitation to newcomers. Due to this, a bad image of organization may get established in the world of open-source communities.
-
To make easy for newcomers and start contributing in just one click, will establish an nice and good image to that open-source enthusiast.
-
From any coala website, the user can choose “Would you like to contribute?” option to join our community.
-
But, the user has to read Newcomers guide before asking for an invite. One of rudimentary check!
-
API Endpoint:
from .views import NewcomerInvite urlpatterns = [ path('^newcomer/invite/', NewcomerInvite.as_view(), name='Newcomer invitation') ]
-
Newcomer invitation class view:
from rest_framework.generics import CreateAPIView from .models import Contributor from .serializers import NewcomerInviteSerializer class NewcomerInvite(CreateAPIView): permission_classes = [] authentication_classes = [] serializer_class = NewcomerInviteSerializer queryset = Contributor.objects.all()
-
Serializer class to perform some validation:
class NewcomerInviteSerializer(serializers.ModelSerializer): class Meta: model = Contributor fields = '__all__' def validate(self, data): # Validate POST data # 1. newcomer_messgae == 'Hello world!' # "Check user has gone through newcomers guide or not" # 2. Valid username # 3. Valid newcomer issue(link provided)(optional) # 4. Get user name and bio from github # 5. set teams to coala_newcomers # 6. Check whether is already a member or not # 7. set status of account to disable # 8. Perform mentioned checks too in issue return data
-
coala has a reward or a surprise gift for newcomers who have completed the newcomers process as mentioned in Newcomer’s guide.
-
This process can be automated by performing some rudimentary checks to promote the newcomer to developer.
-
A newcomer can choose “Promote to Developer” from profile page on community website. The user can apply for this at most 3 times.
-
API Endpoint:
from .views import NewcomerPromotion urlpatterns = [ path('^newcomer/promote/', NewcomerPromotion.as_view(), name='Newcomer promotion') ]
-
Promotion class view:
from rest_framework.generics import UpdateAPIView from .models import Contributor from .serializers import NewcomerPromoteSerializer class NewcomerPromotion(UpdateAPIView): permission_classes = [] authentication_classes = [] serializer_class = NewcomerPromoteSerializer queryset = Contributor.objects.all()
-
Respective Serializer class for validating:
from rest_framework import serializers from .models import Contributor class NewcomerPromoteSerializer(serializers.ModelSerializer): class Meta: model = Contributor fields = '__all__' def validate(self, data): # Validate POST data # Evaluate the google form # 1. Get user latest entry from response spreadsheet # 2. Valid data i.e. # - username # - solved newcomer issue # - solver low difficult issue # - reviewed two PRs return data
-
Validating this data will definitely require some amount of time. This can be achieved in two ways:
-
One way is that user chooses to get promoted. Then, the API call will be made in background and user can’t do any work until the validating process is completed.
-
Other way is create a database model which will have usernames with other information. And, then a cron Job will be executed for all such usernames to validate their information. The job will be executed for only those username who job scheduled was pending.
I would like to go for second one during coding if I get enough time to achieve this.
-
-
To authenticate/login user, I planned to use django-allauth.
-
It won’t allow to check whether a user is member of coala community or not.
-
To check it, after the user gets logged in - a check will be performed using GitHub v3 API. If user is a part of community, it will be redirected to success page otherwise will be redirected to invitation form!
-
Must be thinking “why an open-source organization need to have authentication?”
It’s a valid question and an important question too. The main thought to add implementation for it “is to avoid spamming” because we can’t provide any anonymous user with these rights:
-
Promotion to developer status
-
Getting assigned to an issue
-
Participate as GSoC mentor
-
Extension to non-availability period during GSoC
-
Google forms
-
Add yourself as a past GSoC student
# Non-working code. # A pseudo-workflow of login def login_using_djangoAllauth(): user = django_allauth.Github() # Doesn't work return user def login(): user = login_using_djangoAllauth() # Check user is part of coala newcomers # if true: # success_login() # else: # raise an error and redirect # to newcomer invitation page
-
One of the most important defined objective in this project is the creation of a profile page. Following information/data will be displayed on profile along with some options. In-order to quickly render the page, python multiprocessing library will be made in use. On profile page,
- Following options(enabled/disabled buttons) will be provided -
-
To only those users who are a part of coala-developers or coala-maintainers team
-
Participate as a Mentor
-
Add new google form
-
Add yourself as a GSoC student
-
Add/update a calendar event
# GitHub v3 API usage to check membership def participate_as_mentor(): if user in (coala_developers() or coala_mainatiners()): return True return False
-
-
To all members of community
- Promotion to developer
- Want to work on an issue?
- Request to update my data(will be done via cron job)
-
- Following information/data will be displayed -
-
User statistics in form of graph
-
Current year contributions
- Issues created
- Issues solved
- PRs Review
- No. of commits
-
Repository-wise contributions (Overall or yearly)
# Overall Results # Yearly results can also be obtained by passing # an extra argument. def get_stats(user, repo): commit_count = get_commit_count(user, repo) review_count = get_review_count(user, repo) issues_cr_count = get_issue_cr_count(user, repo) prs_cr_count = get_prs_cr_count(user, repo) return { 'commits': commit_count, 'reviews': review_count, 'issues': issues_cr_count, 'prs': prs_cr_count } repos = get_repos(coala) repo_wise_stats = {} for repo in repos: stats = get_stats(user, repo) repo_wise_stats[repo] = stats
-
Comparison to previous year contributions
-
Issue solved (grouped-by labels)
# Return all issues assigned to user for a given organization # 5-6 labels will be shown in graph where last label will be # names as others issues = get_issues(user, org, state) issue_label_stats = {} for issue in issues: for label in issue.labels(): if label in issue_label_stats: issue_label_stats[label] += 1 else: issue_label_stats[label] = 1
-
-
Activities
-
Rank
-
Personal Information
-
Teams
-
Total time any many more…
-