Frequently Asked Questions
These are questions that have cropped up before, and can hopefully solve your query
'Notebook' seems to mean many things, why?The term is a generic one, and can either apply to 'Notebook App', 'Notebook Dashboard', or 'Notebook document'. See 'What is a computational notebook' below.
Which programming languages are supported on Noteable?Noteable currently supports Python 3 and R. We have experience with notebooks using Haskell, Julia, and Stata kernels. Talk to us about other languages.
Can I share notebook documents/files with other users?Yes, but see the more detailed How can I share notebook documents/files with other users section below.
Can I collaborate with others in my notebook?Yes, you can collaborate with others on your course by sharing a link to your notebook where they can follow your coding. You can share access to a notebook file you are working on using the 'Collaborative session' notebook servers available in Noteable. Note that collaborative sessions use the JupyterLab interface. Find out more information about this feature in the Collaborative Coding section.
Does Noteable support assignments and grading?Yes, Noteable uses nbgrader to provide course-specific assignments, grading, and feedback. See the documentation for Student Guide to Assignments and Instructor Guide to Assignments.
Is Noteable compatible with existing Jupyter notebooks?Yes.
Can I install my own packages on Noteable?Yes, see the detailed How can I install my own packages on Noteable below.
How does Noteable connect to my institution's Learning Management System/Virtual Learning Environment?
Noteable creates a trusted relationship with the LMS/VLE, and thus accepts the details of the authenticated user that LMS/VLE sends over that [authenticated] connection.
- For Scottish schools: Noteable has an agreement in place with the GLOW Digital Learning Platform. Speak to your Local Authority to gain access to Noteable as a free tool in GLOW.
- For VLE/LMS systems using LTI: Contact us through the form on the top-right of the website, and we will send you configuration details that will be specific to your platform.
Connections provide Noteable with five pieces of data:
- The customer (inferred from the trusted configuration)
usernameof the user
- The short-form course code (eg
- The course name (eg "Mathematics 101 - an introduction to Bayesian statistics")
- role ('Instructor' or anything else)
"Customer" and "Username" allow us to provide home directories for each user, and "course" and "role" details are used my the Assignments system to provide appropriate tools.
Can I log into noteable if I don't have access to an LMS/VLE?No. Authentication is always done by the subscribing organisation (but see Trials below)
Can I clear old assignments from my home directory?
Absolutely. The way you do it is slightly different for student & instructors
For a student, you can delete the folder
<course code>/<assignment code>
For an instructor it's more complicated: There is data in your grading database, your original files, and the multiple student submissions & graded files.
The easy solution is to used the
archive-assignment script from a terminal session.
The situation is described in more details in the "Technical questions" section below
Noteable High-Level Design Description
What is a computational notebook?
computational notebook is an electronic 'notebook' designed to allow computation/calculation within the document. Whilst the likes of a word-processor may allow embedded tables and graphs, they don't have the capacity for general-purpose computation. Likewise, a programming editor will allow comments for code, but is not really any good for actually creating narrative notes.
We have to recognise that the term 'notebook' is often over-used, so let's break this down into four parts:
- The 'notebook document' is the file, in
ipynbformat, that contains the (markdown) text, the source code, and the output of your work.
- There's the backend bit that takes the source code and computes it. This is language-specific, and is called the 'kernel'.
- There's the front-end 'notebook dashboard', which is mainly used to open notebook documents and manage files. This is the Classic Notebook Interface, or the Jupyterlab Interface.
- Finally, the 'notebook app' (or 'notebook server') is the term for the behind-the-scenes application that provides the web-server which powers all this.
Where do 'notebooks' come from?
'Notebooks' come from wanting to provide a simple, interactive, programming environment - however to get there, there's some background to cover:
Let's start with 3rd generation programming languages (3GL) - machine independent, abstracted, programmer-centric. The source code is written, then compiled into another form, which is then executed.
3GL programmers often use an integrated development environment (IDE) tool to write, compile, and execute code all from the same place. IDEs often come with additional helpful features such as syntax highlighting, code-completion, and debugging. 3GL programming means a developer can use an IDE on their workstation, with locally installed libraries, to create a compiled application that can be distributed & executed as is on other computers.
iPython took the renowned programming language Python and wrapped an interactive shell around it - so it would compile & execute sections of code in sections. iPython came with a command-line shell, and a game-changing IDE: a web-based interface.
iPython's main goal is not application development, but data analysis, exploration & modelling.
Jupyter Notebooks are a language-agnostic IDE wrapped around iPython, and also bring a wealth of visual & user-centric improvements to the tool. These developments, primarily open-source and developer-community oriented, are the building blocks of the Noteable service.
Where does Noteable come from?
Noteable was born from the want to have notebooks available for teaching, at scale. Providing a 'notebook' for oneself is easy. Providing 'notebooks' for a single class is not difficult. Providing 'notebooks' for a whole University is hard.
Noteable uses Cloud Infrastructure to provide Containerised 'Notebooks' - which thus meets the goals of standardisation, repeatability, and scaling.
What is a docker container?
A Docker container image is a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries and settings.
The important part for us is 'includes everything needed to run an application'.
What does Noteable do?
The problem with a Notebook App is that is runs on the user's computer, and needs the user to have all the libraries installed that they want to use in their notebook documents.
If we can put the Notebook App plus all the desired libraries into a docker container, we have a distributable service that we know is complete & correct.
Noteable is, at this level, two things:
- A mechanism to provide authenticated access to a range of different Docker App containers
- The infrastructure to launch notebook apps into a private, secure cloud
What is the Jupyter Project?The Jupyter Project is a spin-off project from the I-Python project, which initially provided an interface only for the Python language and continues to make available the canonical Python kernel for Jupyter. The name Jupyter itself is derived from the combination of Julia, Python, and R.
What is Jupyterhub?Jupyterhub is a web-application that provides a mechanism for users to connect to a machine and start a containerized Jupyter notebook app. Noteable does not use Jupyterhub.
- Noteable provides the tools and support, for lecturers, students and researchers, whatever their level of skill or knowledge, to deliver narratively rich, coded solutions.
- Noteable enables computational narratives to be readily created, shared and reused within and across teaching and learning contexts.
- In order for data, and the computations that process and visualize that data to be useful and meaningful for humans, they must be embedded into a narrative - a computational narrative - that tells a story for a particular audience and context.
- Our tools are designed by lecturers and researchers to support lecturers and researchers.
- They are easily integrated into your existing Virtual Learning Environment solutions.
- Tailorable to your teaching: Notebooks can be discipline specific or designed to be bespoke for your needs.
- We can support researchers working with data as well: To provide more CPU or more data storage, niche language kernel support and Big Data tooling.
- Your student numbers may change but our infrastructure makes sure your Notebooks still work smoothly. All without recourse to local IT support.
What is 'Trial Access', and can I have one?
We have a Trial Access system for potential customers. By default, users will be given "Instructor"-level access, and will have access to the full service without artifical restriction.
We run a Moodle system specifically for this.
How does my institution's Learning Management System/Virtual Learning Environment connect to Noteable?
Customers can connect to Noteable using a technology called LTI. This places the ownership of Authentication with the subscribing institution, and leaves Authorisation to Noteable. Noteable supports the LTI 1.1 and LTI 1.3 Standards.
Noteable has a parallel system for Scottish Schools, via the GLOW Digital Learning Platform, based on SAML.
It will be useful to assert the terminology used when talking about LTI connections:
LTI 1.1 has a connection from a Consumer (VLE) to a provider (Noteable.)
LTI 1.3 has a connection from a Provider (VLE) to a tool (Noteable.)
Authentication: Confirming the user is who they say they are. (A passport authenticates someone at the airport, but does not allow them access to a flight.)
Authorisation: Asserting the user has access to something. (A boarding pass allows the holder onto a flight, but makes no comment on the identity of the holder.)
Configuration of an LTI 1.1 connection
Noteable will send three pieces of information to the technical contact for the customer:
Key: An identifier unique to your subscription
Secret: A long random sequence of characters
Launch URL: The web address your VLE/LMS connects to, to hand-shake and connect your user.
Configuration of an External Tool varies by VLE/LMS - so we do not comment on that process. Once configured, access to noteable is available.
Please ensure the connection opens in a new window [or tab] - and not stay in an iframe.
Configuration of an LTI 1.3 connection
LTI 1.3 is a more complex system, with a bit more back and forth, but should be a fairly quick process, none-the-less.
Blackboard Ultra (Blackboard LEARN): Noteable is already registered with Blackboard, contact us for the Application ID
All other systems:
Noteable will send four pieces of information to the technical contact for the customer:
Initiate login URL,
Public keyset, and
Configuration of an External Tool varies by VLE/LMS - so we do not comment on that process. Once configured, Noteable requires some information back to complete Authorisation details - eg:
- Platform ID: https://example.com/ultra
- Client ID: 123456AbCd789X
- Deployment ID: 6
- Public keyset URL: https://example.com/ultra/mod/lti/certs
- Access token URL: https://example.com/ultra/mod/lti/token
- Authentication request URL: https://example.com/ultra/mod/lti/auth
For Blackboard Ultra (Blackboard LEARN), you will only have the Deployment ID, the rest are already known to Noteable.
Please ensure the connection opens in a new window [or tab] - and not stay in an iframe.
The process of configuring a VLE/LMS
Generally, this process is handled by the admin team for your VLE/LMS, and only needs to be done once.
We will first set up a connection between your staging/testing service and our noteable-beta service. This will allow us to confirm you can create the connection you want, and to allow your service-team to preview future updates as they come down the line.
We will look for real names (as they get used within the Assignments system), however we do not look for email addresses or any other information not specified here.
LTI is not always consistent in passing data - we would prefer a username such as
35f17232b17b6a4d0025c64b53658088a1545860, and a sensible course code (perhaps
MATH101_2020_SEM1) rather than
86fde15399c4156c8d0aee664b625d3de2920784 - but Noteable will work with the obfuscated codes. Working in Testing/Staging/Beta environments allows these issues to be identified and ironed out before keen instructors jump on board.
Once the noteable-beta connection is working correctly, new configuration details will be used to enable access to the main noteable service.
As a tutor/instructor - for classes
If your VLE/LMS does not have a central configuration for Noteable, you may be able to add it at course-level - however not all VLE/LMS systems allow configuration of a new
External Tool at course level.
Your local VLE/LMS team are best placed to help with this, but essentially you will be adding an
External Tool item to the set of modules & items in the course-content.
We would prefer you to use the following graphic with a link to the noteable service: https://noteable.edina.ac.uk/images/noteable_square_logo.png.
Is there a full list of libraries installed in each notebook?
This assumes the libraries listed in the synopsis descriptions is not enough
Maintaining a complete & comprehensive table of libraries across all notebooks proved to be an enormous, and unwieldy, table - so, no, that list is not available outside the service.
If you start the relevant notebook, you can get the list yourself:
!conda listin a notebook cell, or
conda listin a terminal
installed.packages() will list R libraries.
How can I install my own packages on Noteable?
For most notebooks
In an early cell in your notebook, install the package - use either of these options:
!conda install -y package-name
!pip install package-name
conda is much slower, but checks version dependencies to ensure there are no problems.
pip will just install the library.
In either case - the library will not be present when you next start a notebook server.
You can install Python packages as above (and many r-libraries are available as conda or pip libraries).
For libraries from the R repository, we have found the solution that works is to use the
R -e 'install.packages('testthat', repo='http://cran.rstudio.com/')'will install the
R -e 'install.packages(c('testthat', 'alr3', 'funkyPigeon'), repo='http://cran.rstudio.com/')'will install all three libraries
Packages are installed under
$HOME/R/ and will persist between sessions
How can I share notebook documents/files with other users?
Python-based notebook servers using the Jupyter classic interface include the "
nbgitpuller" extension. If you make files available from an accessible git repo (this does not have to necessarily be GitHub but this is the most popular), then others may download those files. For full details on using this method, please read the nbgitpuller documentation. IMPORTANT: You do NOT need to create fancy links or anything... +GitRepo does that for you.
You can also download and upload files using the Download and Upload functions in the Jupyter classic, JupyterLab and RStudio interfaces.
Most notebook servers have an extension called “nbgrader”. This allows instructors to make assignments available for courses.
Which programming languages are supported on Noteable?
Noteable has different Notebooks available to the user.
The majority of notebooks have
Python 3 with a number of additional libraries pre-installed.
Specialist notebooks provide specialist kernels: specifically
Specialist notebooks provide different Python libraries:
R with Stan, R in the
RStudio interface, ....
Can I have multiple markers?
Yes, but it's not available by default.
The Noteable team can enable Multi-Marker on a per-course basis, or for all courses run by an organisation.
How can I tell if my course has Multi-Marker enabled?
Formgrader, when you click on the name of your assignment, you get taken to a folder to create [or upload] your notebook document(s):
- if that folder is
🖿/source/<assignment-code>, you do not have Multi-Marker enabled
- if that folder is
🖿/instructor-courses/<course-code>/source/<assignment-code>, you do have Multi-Marker enabled
Does Multi-Marker affect the student experience?
Within the Noteable service, the student experience is absolutely the same - with or without Multi-Marker enabled. Multi-Marker only affects Instructors
When can I ask for to be Multi-Marker enabled?
Ask for Multi-Marker before you actually create you first assignment. We recommend all courses switch to having Multi-Marker enabled, even with just one marker.
How can Instructors archive old assignments?
archive-assignment script archives the assignment, and deletes it from your records. It will:
- Manage the database:
- copy the grades for an assignment into a
- delete the assignment-related data from your local
- copy the grades for an assignment into a
- Manage the files:
- create a
.zipfile of all the source, submission, grading, and feedback files that relate to this assignment
- delete them from the disk.
- create a
- Start a new terminal (under the
archive-assignmentto see the options for the command
If we assume the course code is
made up, and assignment is
Test 1, the command would be
archive-assignment "Test 1"
This will create two files:
made up.Test 1.csv and
made up.Test 1.zip
Manually remove all records
The process allows a more controlled, and informed, archiving process. All work is done in a terminal (under the
(These examples assume the course code is
made up, and assignment is
Managing the database
Note: The database does not use the course code to distinguish assignments (which is why you can't repeat assignment names across courses.)
- To list the assignments you [your local database] knows about:
nbgrader db assignment list
- To extract the scores from the database for a given assignment (assume assignment name =
Test 1, and we want to put the contents in a file called
made up.Test 1.csv):
nbgrader export --assignment '["Test 1"]' --to "made up.Test 1.csv"
- To clear the data:
nbgrader db assignment remove "Test 1"
- (It will complain if the assignment has scores/grades recorded)
nbgrader db assignment remove "Test 1" --force
Note: Files use the course code in some file paths
- Check what files you have:
ls source/"Test 1" release/"Test 1" autograded/*/"Test 1" made\ up/collected/*/"Test 1" feedback/*/"Test 1"
- If that looks reasonable, consider making a zip file of all the files:
zip -r "made up.Test 1.zip" source/"Test 1" release/"Test 1" autograded/*/"Test 1" "made up"/collected/*/"Test 1" feedback/*/"Test 1"
- Delete all the files (warning there is no recovery from this):
rm -rf source/"Test 1" release/"Test 1" autograded/*/"Test 1" "made up"/collected/*/"Test 1" feedback/*/"Test 1"