YouTube will demonetize channels that promote anti-vaccination views, after a report by BuzzFeed News found ads, including from health companies, running before anti-vax videos. The platform will also place a new information panel that links to the Wikipedia entry on “vaccine hesitancy” before anti-vax videos. Information panels (part of YouTube’s efforts to combat misinformation) about […]

Bassem Hamdy has been in the construction business for a long time. He spent the last few years at the construction software business Procore, now a $3 billion company developing technology for the construction industry, and now Hamdy is ready to unveil his next act as chief executive and co-founder of Briq, a new software […]

Ev Wiliams is stepping down from Twitter's board of directors effective at the end of the month, according to documents submitted to the Securities and Exchange Commission on Friday.

Relationships between landlords and their tenants don’t need to be fraught ones. With Y Combinator -backed Latchel, landlords and property managers can access a 24/7 maintenance service that takes requests from tenants and deploys the right professional to fix the things that need fixing. “At first we thought of doing a high-tech property management system,” Latchel founder […]

Tesla and SpaceX CEO Elon Musk has been teasing — and his fan base has been making pleas — to host a meme review. And after tweeted hints, meme review has arrived via YouTube star PewDiePie. Musk tweeted last month a photo and a question “Host meme review?” Host meme review? pic.twitter.com/k2SFtIUh1k — Elon Musk […]

Apple has confirmed its plans to close retail stores in the Eastern District of Texas — a move that will allow the company to better protect itself from patent infringement lawsuits, according to Apple news sites 9to5Mac and MacRumors, which broke the news of the stores’ closures. Apple says that the impacted retail employees will […]

The video conferencing company Zoom is aiming to file a public S-1 by the end of March, according to a new report in Business Insider that adds the company could go public as soon as April. Business Insider reported last month that Zoom had filed confidentially with the SEC to go public, just months after […]

In this article I'll share some personal thoughts about Test Automation Frameworks; you can take inspiration from them if you are going to evaluate different test automation platforms or assess your current test automation solution (or solutions).Despite it is a generic article about test automation, you'll find many examples explaining how to address some common needs using the Python based test framework named pytest and the Jenkins automation server: use the information contained here just as a comparison and feel free to comment sharing alternative methods or ideas coming from different worlds.It contains references to some well (or less) known pytest plugins or testing libraries too.Before talking about automation and test automation framework features and characteristics let me introduce the most important test automation goal you should always keep in mind.Test automation goals: ROIYou invest in automation for a future return of investment.Simpler approaches let you start more quickly but in the long term they don't perform well in terms of ROI and vice versa. In addition the initial complexity due to a higher level of abstraction may produce better results in the medium or long term: better ROI and some benefits for non technical testers too. Have a look at the test automation engineer ISTQB certification syllabus for more information:https://www.istqb.org/downloads/category/48-advanced-level-test-automation-engineer-documents.htmlSo what I mean is that test automation is not easy: it doesn't mean just recording some actions or write some automated test procedures because how you decide to automate things affects the ROI. Your test automation strategy should consider your tester technical skills now and future evolutions, considerations about how to improve your system testability (is your software testable?), good test design and architecture/system/domain knowledge. In other words be aware of vendors selling "silver bullet" solutions promising smooth test automation for everyone, especially rec&play solutions: there are no silver bullets.Test automation solution features and characteristicsA test automation solution should be enough generic and flexible, otherwise there is the risk of having to adopt different and maybe incompatible tools for different kind of tests. Try to imagine the mess of having the following situation: one tool or commercial service for browser based tests only based on rec&play, one tool for API testing only, performance test frameworks that doesn't let you reuse existing scenarios, one tool for BDD only scenarios, different Jenkins jobs with different settings for each different tool, no test management tool integration, etc. A unique solution, if possible, would be better: something that let you choose the level of abstraction and that doesn't force you. Something that let you start simple and that follow your future needs and the skill evolution of your testers.That's one of the reasons why I prefer pytest over an hyper specialized solution like behave for example: if you combine pytest+pytest-bdd you can write BDD scenarios too and you are not forced to use a BDD only capable test framework (without having the pytest flexibility and tons of additional plugins).And now, after this preamble, an unordered list of features or characteristics that you may consider for your test automation solution software selection:fine grained test selection mechanism that allows to be very selective when you have to choose which tests you are going to launchparametrizationhigh reusetest execution logs easy to read and analyzeeasy target environment switchblock on first failurerepeat your tests for a given amount of timesrepeat your tests until a failure occurssupport parallel executionsprovide integration with third party software like test management toolsintegration with cloud services or browser gridsexecute tests in debug mode or with different log verbositysupport random tests execution order (the order should be reproducible if some problems occur thanks to a random seed if needed)versioning supportintegration with external metrics engine collectorssupport different levels of abstraction (e.g., keyword driven testing, BDD, etc)rerun last failedintegration with platforms that let you test against a large combination of OS and browsers if neededare you able to extend your solution writing or installing third party plugins? Typically a test automation engineer will be able to drive automated test runs using the framework command line interface (CLI) during test development but you'll find out very soon that you need an automation server for long running tests, scheduled builds, CI and here it comes Jenkins. Jenkins could be used by non technical testers for launching test runs or initialize an environment with some test data.JenkinsWhat is Jenkins? From the Jenkins website:Continuous Integration and Continuous Delivery. As an extensible automation server, Jenkins can be used as a simple CI server or turned into the continuous delivery hub for any project.So thanks to Jenkins everyone can launch a parametrized automated test session just using a browser: no command line and nothing installed on your personal computer. So more power to non technical users thanks to Jenkins!With Jenkins you can easily schedule recurrent automatic test runs, start remotely via external software some parametrized test runs, implement a CI and many other things. In addition as we will see Jenkins is quite easy to configure and manage thanks to through the web configuration and/or Jenkins pipelines.Basically Jenkins is very good at starting builds and generally jobs. In this case Jenkins will be in charge of launching our parametrized automated test runs.And now let's talk a little bit of Python and the pytest test framework.Python for testing I don't know if there are some articles talking about statistics on the net about the correlation between Test Automation Engineer job offers and the Python programming language, with a comparison between other programming languages. If you find a similar resource share with me please!My personal feeling observing for a while many Test Automation Engineer job offers (or any similar QA job with some automation flavor) is that the Python word is very common. Most of times is one of the nice to have requirements and other times is mandatory.Let's see why the programming language of choice for many QA departments is Python, even for companies that are not using Python for building their product or solutions.Why Python for testingWhy Python is becoming so popular for test automation? Probably because it is more affordable for people with no or little programming knowledge compared to other languages. In addition the Python community is very supportive and friendly especially with new comers, so if you are planning to attend any Python conference be prepared to fall in love with this fantastic community and make new friends (friends, not only connections!). For example at this time of writing you are still in time for attending PyCon Nove 2018 in the beautiful Florence (even better if you like history, good wine, good food and meet great people):  https://www.pycon.itYou can just compare the most classical hello world, for example with Java:public class HelloWorld {     public static void main(String[] args) {         System.out.println("Hello, World!");    } }and compare it with the Python version now:print("Hello, World!")Do you see any difference? If you are trying to explain to a non programmer how to print a line in the terminal window with Java you'll have to introduce public, static, void, class, System, installing a runtime environment choosing from different version, installing an IDE, running javac, etc and only at the end you will be able to see something printed on the screen. With Python, most of times it comes preinstalled in many distributions, you just focus on what to need to do. Requirements: a text editor and Python installed. If you are not experienced you start with a simple approach and later you can progressively learn more advanced testing approaches.And what about test assertions? Compare for example a Javascript based assertions:expect(b).not.toEqual(c);with the Python version:assert b != cSo no expect(a).not.toBeLessThan(b), expect(c >= d).toBeTruthy() or expect(e).toBeLessThan(f): with Python you just say assert a >= 0 so nothing to remember for assertions!Python is a big fat and very powerful programming language but it follows a "pay only for what you eat" approach.Why pytestIf Python is the language of your choice you should consider the pytest framework and its high quality community plugins and I think it is a good starting point for building your own test automation solution. The pytest framework (https://docs.pytest.org/en/latest/) makes it easy to write small tests, yet scales to support complex functional testing for applications and libraries.Most important pytest features:simple assertions instead of inventing assertion APIs (.not.toEqual or self.assert*)auto discovery test modules and functionseffective CLI for controlling what is going to be executed or skipped using expressionsfixtures, easy to manage fixtures lifecycle for long-lived test resources and parametrized features make it easy and funny implementing what you found hard and boring with other frameworksfixtures as function arguments, a dependency injection mechanism for test resourcesoverriding fixtures at various levelsframework customizations thanks to pluggable hooksvery large third party plugins ecosystemI strongly suggest to have a look at the pytest documentation but I'd like to make some examples showing something about fixtures, code reuse, test parametrization and improved maintainability of your tests. If you are not a technical reader you can skip this section.I'm trying to explain fixtures with practical examples based on answers and questions:When should be created a new instance of our test resource?You can do that with the fixture scope (session, module, class, function or more advanced options like autouse). Session means that your test resource will live for the entire session, module/class for all the tests contained in that module or class, with function you'll have an always fresh instance of your test resource for each testHow can I determine some teardown actions at the end of the test resource life?You can add a sort of fixture finalizer after the yield line that will be invoked at the end of our test resource lifecycle. For example you can close a connection, wipe out some data, etc.How can I execute all my existing tests using that fixture as many as your fixture configurations?You can do that with params. For example you can reuse all your existing tests verifying the integration with different real databases, smtp servers. Or if you have the web application offering the same features deployed with a different look&feel for different brands you can reuse all your existing functional UI tests thanks to pytest's fixture parametrization and a page objects pattern where for different look&feel I don't mean only different CSS but different UI components (e.g. completely different datetime widgets or navigation menu), components disposition in page, etc.How can I decouple test implementation and test data? Thanks to parametrize you can decouple them and write just one time your test implementation. Your test will be executed as many times as your different test dataHere you can see an example of fixture parametrization (the test_smtp will be executed twice because you have 2 different fixture configurations): import pytestimport smtplib@pytest.fixture(scope="module",                        params=["smtp1.com", "smtp2.org"])def smtp(request):    smtp = smtplib.SMTP(request.param, 587, timeout=5)    yield smtp    print("finalizing %s" % smtp)    smtp.close() def test_smtp(smtp):    # use smtp fixture (e.g., smtp.sendmail(...))    # and make some assertions.     # The same test will be executed twice (2 different params)    ... And now an example of test parametrization:import pytest@pytest.mark.parametrize("test_input,expected", [    ("3+5", 8),    ("2+4", 6),    ("6*9", 42), ])def test_eval(test_input, expected):    assert eval(test_input) == expectedFor more info see:https://docs.pytest.org https://docs.pytest.org/en/latest/fixture.htmlhttps://docs.pytest.org/en/latest/fixture.html#parametrizing-fixtureshttps://docs.pytest.org/en/latest/parametrize.htmlThis is only pytest, as we will see there are many pytest plugins that extend the pytest core features.Pytest pluginsThere are hundreds of pytest plugins, the ones I am using more frequently are: pytest-bdd, BDD library for the pytest runner pytest-variables, plugin for pytest that provides variables to tests/fixtures as a dictionary via a file specified on the command line pytest-html, plugin for generating HTML reports for pytest results pytest-selenium, plugin for running Selenium with pytest pytest-splinter, a pytest-selenium alternative based on Splinter. pPytest splinter and selenium integration for anyone interested in browser interaction in tests pytest-xdist, a py.test plugin for test parallelization, distributed testing and loop-on-failures testing modes pytest-testrail, pytest plugin for creating TestRail runs and adding results on the TestRail test management toolpytest-randomly, a pytest plugin to randomly order tests and control random seed (but there are different random order plugins if you search for "pytest random")pytest-repeat, plugin for pytest that makes it easy to repeat a single test, or multiple tests, a specific number of times. You can repeat a test or group of tests until a failure occurspytest-play, an experimental rec&play pytest plugin that let you execute a set of actions and assertions using commands serialized in JSON format. Makes test automation more affordable for non programmers or non Python programmers for browser, functional, API, integration or system testing thanks to its pluggable architecture and many plugins that let you interact with the most common databases and systems. It provides also some facilitations for writing browser UI actions (e.g., implicit waits before interacting with an input element) and asynchronous checks (e.g., wait until a certain condition is true)Python libraries for testing:PyPOM, python page object model for Selenium or Splinter  pypom_form, a PyPOM abstraction that extends the page object model applied to forms thanks to declarative form schemas Scaffolding tools:cookiecutter-qa, generates a test automation project ready to be integrated with Jenkins and with the test management tool TestRail that provides working hello world examples. It is shipped with all the above plugins and it provides examples based on raw splinter/selenium calls, a BDD example and a pytest-play example  cookiecutter-performance, generate a tox based environment based on Taurus bzt for performance test. BlazeMeter ready for distributed/cloud performance tests. Thanks to the bzt/taurus pytest executor you will be able to reuse all your pytest based automated tests for performance testsPytest + Jenkins togetherWe've discussed about Python, pytest and Jenkins main ingredients for our cocktail recipe (shaken, not stirred). Optional ingredients: integration with external test management tools and selenium grid providers.Thanks to pytest and its plugins you have a rich command line interface (CLI); with Jenkins you can schedule automated builds, setup a CI, let not technical users or other stakeholders executing parametrized test runs or building test always fresh test data on the fly for manual testing, etc. You just need a browser, nothing installed on your computer.Here you can see how our recipe looks like:Now lets comment all our features provided by the Jenkins "build with parameters" graphical interface, explaining option by option when and why they are useful.Target environment (ENVIRONMENT)In this article we are not talking about regular unit tests, the basis for your testing pyramid. Instead we are talking about system, functional, API, integration, performance tests to be launched against a particular instance of an integrated system (e.g., dev, alpha or beta environments).You know, unit tests are good they are not sufficient: it is important to verify if the integrated system (sometimes different complex systems developed by different teams under the same or third party organizations) works fine as it is supposed to do. It is important because it might happen that 100% unit tested systems doesn't play well after the integration for many different reasons. So with unit tests you take care about your code quality, with higher test levels you take care about your product quality. Thanks to these tests you can confirm an expected product behavior or criticize your product.So thanks to the ENVIRONMENT option you will be able to choose one of the target environments. It is important to be able to reuse all your tests and launch them against different environments without having to change your testware code. Under the hood the pytest launcher will be able to switch between different environments thanks to the pytest-variables parametrization using the --variables command line option, where each available option in the ENVIRONMENT select element is bound to a variables files (e.g., DEV.yml, ALPHA.yml, etc) containing what the testware needs to know about the target environment.Generally speaking you should be able to reuse your tests without any modification thanks to a parametrization mechanism.If your test framework doesn't let you change target environment and it forces you to modify your code, change framework.Browser settings (BROWSER)This option makes sense only if you are going to launch browser based tests otherwise it will be ignored for other type of tests (e.g., API or integration tests).You should be able to select a particular version of browser (latest or a specific version) if any of your tests require a real browser (not needed for API tests just for making one example) and preferably you should be able to integrate with a cloud system that allows you to use any combination of real browsers and OS systems (not only a minimal subset of versions and only Firefox and Chrome like several test platforms online do). Thanks to the BROWSER option you can choose which browser and version use for your browser based tests. Under the hood the pytest launcher will use the --variables command line option provided by the pytest-variables plugin, where each option is bound to a file containing the browser type, version and capabilities (e.g., FIREFOX.yml, FIREFOX-xy.yml, etc). Thanks to pytest, or any other code based testing framework, you will be able to combine browser interactions with non browser actions or assertions.A lot of big fat warnings about rec&play online platforms for browser testing or if you want to implement your testing strategy using only or too many browser based tests. You shouldn't consider only if they provide a wide range of OS and versions, the most common browsers. They should let you perform also non browser based actions or assertions (interaction with queues, database interaction, http POST/PUT/etc calls, etc). What I mean is that sometimes only a browser is not sufficient for testing your system: it might be good for a CMS but if you are testing an IoT platform you don't have enough control and you will write completely useless tests or low value tests (e.g., pure UI checks instead of testing reactive side effects depending on eternal triggers, reports, device activity simulations causing some effects on the web platform under test, etc).In addition be aware that some browser based online testing platforms doesn't use Selenium for their browser automation engine under the hood. For example during a software selection I found an online platform using some Javascript injection for implementing user actions interaction inside the browser and this might be very dangerous. For example let's consider a login page that takes a while before the input elements become ready for accepting the user input when some conditions are met. If for some reasons a bug will never unlock the disabled login form behind a spinner icon, your users won't be able to login to that platform. Using Selenium you'll get a failing result in case of failure due to a timeout error (the test will wait for elements won't never be ready to interact with and after few seconds it will raise an exception) and it's absolutely correct. Using that platform the test was green because under the hood the input element interaction was implemented using DOM actions with the final result of having all your users stuck: how can you trust such platform?OS settings (OS)This option is useful for browser based tests too. Many Selenium grid vendors provide real browser on real OS systems and you can choose the desired combination of versions.Resolution settings (RESOLUTION)Same for the above options, many vendor solutions let you choose the desired screen resolution for automated browser based testing sessions.Select tests by names expressions (KEYWORDS)Pytest let you select the tests you are going to launch selecting a subset of tests that matches a pattern language based on test and module names.For example I find very useful to add the test management tool reference in test names, this way you will be able to launch exactly just that test:c93466Or for example all test names containing the login word but not c92411:login and not c92411Or if you organize your tests in different modules you can just specify the folder name and you'll select all the tests that live under that module:api Under the hood the pytest command will be launched with -k "EXPRESSION", for example-k "c93466"It is used in combination with markers, a sort of test tags. Select tests to be executed by tag expressions (MARKERS)Markers can be used alone or in conjunction with keyword expressions. They are a sort of tag expression that let you select just the minimum set of tests for your test run.Under the hood the pytest launcher uses the command line syntax -m "EXPRESSION".For example you can see a marker expression that selects all tests marked with the edit tag excluding the ones marked with CANBusProfileEdit: edit and not CANBusProfileEditOr execute only edit negative tests:  edit and negativeOr all integration tests integrationIt's up to you creating granular keywords for features and all you need for select your tests (e.g., functional, integration, fast, negative, ci, etc).Test management tool integration (TESTRAIL_ENABLE)All my tests are decorated with the test case identifier provided by the test management tool, in my company we are using TestRail.If this option is enabled the test results of executed tests will be reported in the test management tool.Implemented using the pytest-testrail plugin.Enable debug mode (DEBUG)The debug mode enables verbose logging.In addition for browser based tests open selenium grid sessions activating debug capabilities options (https://www.browserstack.com/automate/capabilities). For example verbose browser console logs, video recordings, screenshots for each step, etc. In my company we are using a local installation of Zalenium and BrowserStack automate.Block on first failure (BLOCK_FIRST_FAILURE)This option is very useful for the following needs:a new build was deployed and you want to stop on the very first failure for a subset of sanity/smoke testsyou are launching repeated, long running, parallel tests and you want to block on first failureThe first usage let you gain confidence with a new build and you want to stop on the very first failure for analyzing what happened.The second usage is very helpful for:random problems (playing with number of repeated executions, random order and parallelism you can increase the probability of reproducing a random problem in less time)memory leakstesting system robustness, you can stimulate your system running some integration tests sequentially and then augment the parallelism level until your local computer is able to sustain the load. For example launching 24+ parallel integration tests on a simple laptop with pytest running on virtual machine is still fine. If you need something more heavy you can use distribuited pytest-xdist sessions or scale more with BlazeMeterAs you can imagine you may combine this option with COUNT, PARALLEL_SESSIONS, RANDOM_ENABLE and DEBUG depending on your needs. You can test your tests robustness too.Under the hood implemented using the pytest's -x option.Parallel test executions (PARALLEL_SESSIONS)Under the hood implemented with pytest-xdist's command line option called -n NUM and let you execute your tests with the desired parallelism level.pytest-xdist is very powerful and provides more advanced options and network distributed executions. See https://github.com/pytest-dev/pytest-xdist for further options.Switch from different selenium grid providers (SELENIUM_GRID_URL)For browser based testing by default your tests will be launched on a remote grid URL. If you don't touch this option the default grid will be used (a local Zalenium or any other provider) but in case of need you can easily switch provider without having to change nothing in your testware.If you want you can save money maintaining and using a local Zalenium as default option; Zalenium can be configured as a selenium grid router that will dispatch capabilities that it is not able to satisfy. This way you will be able to save money and augment a little bit the parallelism level without having to change plan.Repeat test execution for a given amount of times (COUNT)Already discussed before, often used in conjunction with BLOCK_FIRST_FAILURE (pytest core -x option) If you are trying to diagnose an intermittent failure, it can be useful to run the same test or group of tests over and over again until you get a failure. You can use py.test's -x option in conjunction with pytest-repeat to force the test runner to stop at the first failure.Based on pytest-repeat's --count=COUNT command line option. Enable random test ordering execution (RANDOM_ENABLE)This option enables random test execution order.At the moment I'm using the pytest-randomly plugin but there are 3 or 4 similar alternatives I have to try out.By randomly ordering the tests, the risk of surprising inter-test dependencies is reduced.Specify a random seed (RANDOM_SEED)If you get a failure executing a random test, it should be possible to reproduce systematically rerunning the same tests order with the same test data.Always from the pytest-randomly readme:By resetting the random seed to a repeatable number for each test, tests can create data based on random numbers and yet remain repeatable, for example factory boy’s fuzzy values. This is good for ensuring that tests specify the data they need and that the tested system is not affected by any data that is filled in randomly due to not being specified. Play option (PLAY)This option will be discussed in a dedicated blog post I am going to write.Basically you are able to paste a JSON serialization of actions and assertions and the pytest runner will be able to execute your test procedure.You need just a computer with a browser for running any test (API, integration, system, UI, etc). You can paste how to reproduce a bug on a JIRA bug and everyone will be able to paste it on the Jenkins build with parameters form.See pytest-play for further information.If you are going to attending next Pycon in Florence don't miss the following pytest-play talk presented by Serena Martinetti:https://www.pycon.it/conference/talks/integration-tests-ready-to-use-with-pytest-playUPDATES:[20180424] pytest-play introduction http://davidemoro.blogspot.it/2018/04/hello-pytest-play.html[20180222] pytest-play 2.0.0 released, no more compatible with 1.x format (using YAML instead of JSON and other improvements). See a detailed article showing how to use pytest-play for API REST testing using plain YAML files https://davidemoro.blogspot.com/2019/02/api-rest-testing-pytest-play-yaml-chuck-norris.html How to create a pytest projectIf you are a little bit curious about how to install pytest or create a pytest runner with Jenkins you can have a look at the following scaffolding tool:cookiecutter-qaIt provides a hello world example that let you start with the test technique more suitable for you: plain selenium scripts, BDD or pytest-play JSON test procedures. If you want you can install page objects library. So you can create a QA project in minutes.Your QA project will be shipped with a Jenkinsfile file that requires a tox-py36 docker executor that provides a python3.6 environment with tox already installed; unfortunately tox-py36 is not yet public so you should implement it by your own at the moment.Once you provide a tox-py36 docker executor the Jenkinsfile will create for you the build with parameters Jenkins form for you automatically on the very first Jenkins build for your project.ConclusionsI hope you'll find some useful information in this article: nice to have features for test frameworks or platform, a little bit of curiosity for the Python world or  new pytest plugin you never heard about.Feedback and contributions are always welcome.Tweets about test automation and new articles happens here:@davidemoro

pytest-play is a rec&play (rec not yet available) pytest plugin that let you execute a set of actions and assertions using commands serialized in JSON format. It tries to make test automation more affordable for non programmers or non Python programmers for browser, functional, API, integration or system testing thanks to its pluggable architecture and third party plugins that let you interact with the most common databases and systems.In addition it provides also some facilitations for writing browser UI actions (e.g., implicit waits before interacting with an input element. The Cypress framework for me was a great source of inspiration) and asynchronous checks (e.g., wait until a certain condition is true).You can use pytest-play programmatically (e.g., use the pytest-play engine as a library for pytest-play standalone scenarios or using the pytest-play API implementing BDD steps).Starting from pytest-play>1.4.x it was introduced a new experimental feature that let you use pytest-play as a framework creating Python-free automated tests based on a JSON based serialization format for actions and assertions (in the next future the more user friendly YAML format will be supported).So now depending on your needs and skills you can choose to use pytest-play as a library or as a framework.In this article I'm going to show how to implement a Plone CMS based login test using the python-free approach without having to write any line of Python code.What is pytest-play and why it existsIn this section I'm going to add more information about the pytest-play approach and other considerations: if you want to see now how to implement our Python-free automated login test jump to the next section!Hyper specialized tool problemsThere are many commercial products or tools that offer solutions for API testing only, browser testing only. Sometimes hyper specialized tools might fit your needs (e.g., a content management system based web application) but sometimes they are not helpful for other distributed applications.For example an API-only platform is not effective for testing a CQRS based application. It is not useful testing only HTTP 200 OK response, you should test that all the expected commands are generated on the event store (e.g., Cassandra) or other side effects.Another example for an IoT applications and UI/browser only testing platforms. You cannot test reactive web apps only with a browser, you should control also simulated device activities (e.g., MQTT, queues, API) for messages/alarms/reports) or any other web based interactions performed by other users (e.g., HTTP calls); you might need to check asynchronously the expected results on web sockets instead of using a real browser implementing when some actions are performed.What is pytest-play In other words pytest-play is an open source testing solution based on the pytest framework that let you:write actions and cross assertions against different protocols and test levels in the same scenario (e.g., check HTTP response and database assertions)minimize the development of Selenium-based asynchronous wait functions before interacting with input elements thanks to implicit wait functions that let you interact with web elements only when they are ready. You just focus on user actions, you are more productive and you reduce the chance of writing bad or not robust asynchronous wait functionsimplement polling-based asynchronous waiter commands based on custom expressions when neededusing a serialization format (JSON at this time of writing, YAML in the next future) that should be more affordable for non technical testers, non programmers or programmers with no Python knowledge.Potentially you will be able to share and execute a new scenario not yet included in your test library copying and pasting a pytest-play JSON to a Jenkins build with parameters form like the following one (see the PLAY textarea):From http://davidemoro.blogspot.it/2018/03/test-automation-python-pytest-jenkins.htmlIn addition if you are a technical user you can extend it writing your own plugins, you can provide the integration with external tools (e.g., test management tools, software metrics engines, etc), you can decide the test abstraction depending on deadlines/skills/strategy (e.g., use plain json files, a programmatic approach based on json scenarios or BDD steps based on pytest-play).What pytest-play is notFor example pytest-play doesn't provide a test scenario recorder but it enforces users to understand what they are doing.It requires a very very little programming knowledge for writing some assertions using simple code expressions but with a little training activity it is still affordable by non programmers (you don't have to learn a programming language, just some basic assertions).It is not feature complete but it is free software.If you want to know more in this previous article I've talked about:test automation framework thoughts plus a comprehensive, detailed look at using pytest and Jenkins for automated testing. A pytest-play example: parametrized login (featuring Plone CMS)In this example we'll see how to write and execute pure json pytest-play scenarios with test data decoupled by the test implementation and test parametrization. I'm using the available online Plone 5 demo site kindly hosted by Andreas Jung (www.zopyx.com).The project is available here:https://github.com/davidemoro/pytest-play-plone-exampleThe tests could be launched this way as a normal pytest project once you installed pytest and the dependencies (there is a requirements.txt file, see the above link):$ pytest --variables env-ALPHA.yml --splinter-webdriver firefox --splinter-screenshot-dir /tmp -xWhere the you can have multiple environment/variable files. E.g., env-ALPHA.yml containing the alpha base url and any other variables:pytest-play: base_url: https://plone-demo.infoOur login test_login.json scenario contains (as you can see there are NO asynchronous waits because they are not needed for basic examples so you can focus on actions and assertions thanks to implicit waits):{ "steps": [ { "comment": "visit base url", "type": "get", "url": "$base_url" }, { "comment": "click on login link", "locator": { "type": "id", "value": "personaltools-login" }, "type": "clickElement" }, { "comment": "provide a username", "locator": { "type": "id", "value": "__ac_name" }, "text": "$username", "type": "setElementText" }, { "comment": "provide a password", "locator": { "type": "id", "value": "__ac_password" }, "text": "$password", "type": "setElementText" }, { "comment": "click on login submit button", "locator": { "type": "css", "value": ".pattern-modal-buttons > input[name=submit]" }, "type": "clickElement" }, { "comment": "wait for page loaded", "locator": { "type": "css", "value": ".icon-user" }, "type": "waitForElementVisible" } ]} Plus an optional test scenario metadata file test_login.ini that contains pytest keyword and decoupled test data: [pytest]markers = logintest_data = {"username": "siteadmin", "password": "siteadmin"} {"username": "editor", "password": "editor"} {"username": "reader", "password": "reader"}Thanks to the metadata file you have just one scenario and it will be executed 3 times (as many times as test data rows)!Et voilà, let's see it in action out scenario without having to write any line of Python code:  There is only a warning I have to remove but it worked and we got exactly 3 different test runs for our login scenario as expected! pytest-play statuspytest-play should be still considered experimental software and many features needs to be implemented or refactored:yaml instead of json. YAML will become the primary configuration format (it should be more user friendly as suggested by some users)API should not be considered stable until future 2.x version improve API testing when using pure json scenarios registering functions (e.g., invoke a function returning a valid authentication bearer for authenticated API testing)implement some python requests library features not yet implemented in play_requests (e.g., cookies)refactor parametrization and templating (Jinja?)implement additional Selenium actions (e.g., right clicks, upload files, etc) implement other cool Cypress ideas enabling non expert testers in writing more robust Selenium scenariosadd page object abstraction in pytest-play based Selenium scenarios with new commands that let you interact with page regions and interact with complex UI widgetsownership change, waiting for pytest-dev core developers approval. Probably soon the ownership will change from davidemoro/pytest-play to pytest-dev/pytest-play once the approvation iter will finish PyCon Nove @ FlorenceIf you are going to attending next PyCon Nove in Florence don't miss the following pytest-play talk presented by Serena Martinetti:https://www.pycon.it/conference/talks/integration-tests-ready-to-use-with-pytest-playDo you like pytest-play?Tweets about pytest-play happens on @davidemoro.Positive or negative feedback is always appreciated. If you find interesting the concepts behind pytest-play let me know with a tweet, add a new pytest-play adapter and/or add a GitHub star if you liked it:Star Updates[20190222] pytest-play 2.0.0 released, no more backwards compatible with 1.x format. Using YAML syntax instead of JSON now and many other improvements and new features. See https://davidemoro.blogspot.com/2019/02/api-rest-testing-pytest-play-yaml-chuck-norris.html

On behalf of the Nikola team, I am pleased to announce the immediate availability of Nikola v8.0.2. This is a quality-of-life release with a handful of bug fixes, two new translations and a few extra features. What is Nikola? Nikola is a static site and blog generator, written in Python. It can use Mako and Jinja2 templates, and input in many popular markup formats, such as reStructuredText and Markdown — and can even turn Jupyter Notebooks into blog posts! It also supports image galleries, and is multilingual. Nikola is flexible, and page builds are extremely fast, courtesy of doit (which is rebuilding only what has been changed). Find out more at the website: https://getnikola.com/ Downloads Install using pip install Nikola. (Python 3-only since v8.0.0.) Changes Make ARCHIVE_PATH, ARCHIVE_FILENAME translatable (Issue #3234) Support configuring Isso via GLOBAL_CONTEXT['isso_config'] (Issue #3225) Handle fragments in doc role (Issue #3212) Slugify references in doc role. Add Interlingua translation by Alberto Mardegan Add Afrikaans translation by Friedel Wolff Support for docutils.conf (Issue #3188) Bugfixes Avoid random rebuilds with sites whose locales are not fullly supported, and random rebuilds on multilingual sites using Python 3.4/3.5 (Issue #3216) Apply modifications to default_metadata before copying it to other languages Make Commento comments work (Issue #3198) Correctly handle separators in the relative path given to "ignored_assets" key in theme meta files (Issue #3210) Fix error when nikola new_post receives directory name as path (Issue #3207) Add slashes to paths with query strings in nikola serve only if there isn’t one before ? Read listings files as UTF-8 Set one-file status basing on default language only (Issue #3191) Don’t warn if post status is set to published explicitly (Issue #3181) Remove mention of Twitter cards requiring an opt-in. This is not true anymore - anyone can use them. fancydates now workwith listdate items (eg. archives) bootstrap4 and bootblog4 themes no longer load moment.js when fancydates are off. (Issue #3231)

I have a particular dislike for nonspecific negativity, where nothing can be done to address its source because the reasons underlying it are never explicitly described. In the context of Mitogen, there has been a consistent stream of this sort originating from an important camp in public spaces, and despite efforts to bring specifics out into the open, still it continues to persist. For that reason I'd like to try a new strategy: justify the negativity and give it a face by providing all the fuel it needs to burn. Therefore in this post, in the interests of encouraging honesty, I will critique my own work. Mitogen is threaded Mitogen is a prehistoric design, with the earliest code dating all the way back to 2007, a time when threading seemed cool and the 'obvious' solution to the problem the library intended to solve. Through mountains of words I have since then justified the use of threading, as if the use of threads were a mandatory feature. In some scenarios this is partially true, but just as often entirely incorrect. This code does everything your mother told you never to do when using threads, and suffers endless asynchrony issues as a result. Historically it failed to lock mutable data structures, and some remnants of that past remain to the present day. If you're going to throw tomatoes, threading is a great place to start. If there is any legitimate reason to have a problem with integrating this library, look no further. The documentation sucks If this guy gets hit by a bus, what happens to the library? Valid concern! Documentation is currently inconsistent at best, out of date and nonexistent at worst. I care a lot about good documentation, but not quite as much as keeping users happy. Therefore as a result, a description of the internals detailed enough to allow someone to take over the work, or even fix some kinds of bugs, does not as yet exist. It's not that Mitogen is exactly rocket science to maintain, but through poor and inconcise documentation, coupled with a mysterious name, a simple solution is cloaked in a mysterious aura of inscrutability. That incites fear in many who attempt to grasp it. Mitogen isn't tested properly Due to the endless asynchrony issues created by threading and running across multiple processes, Mitogen is a nightmare to test. In the common case where some important test passes, there is little to no guarantee that test will continue to pass, say, if the moon turned blue or a flock of migratory pink flamingos happened to pass overhead. Addressing testing in Mitogen is more than a full-time job, it's an impossible nightmare literally exceeding the boundaries of computer science. Due to the library's design, and the underlying use of threading, a combinatorial explosion of possible program states are produced for which no good testing strategy can or ever will exist. Many of these aspects can be tested only partially by throwing the library at the wall in a loop and ensuring it sticks, and due to the problem it tries to solve, this will likely remain true in perpetuity. The sound byte you're looking for to justify your loathing is "Mitogen is spaghetti code that's impossible to test" Mitogen does things it shouldn't do. If you look closely enough you'll find some modules importing ctypes and doing all kinds of crazy things to work around deficiencies in Python's interfaces. Who the hell writes cowboy code like this? What happens if I run this on some ancient 286 running Xenix? While such atrocities are carefully sandboxed, the reality is that testing in these scenarios has never been done. The computer might catch fire, or the moon might coming crashing into the earth. Mitogen is riddled with internationalization issues As a side effect of being written by a monolingual developer, Mitogen sometimes doesn't have a clue about sudo or su when it says things like المصادقة غير صحيحة, therefore you can never really truly 100% rely on it to function when charging handsomely to peddle someone else's work at customer sites, when some of those might be large international concerns, say, in the oil or finance sector. Because of the hacky methods used to integrate external tools, there may always be some operating system, tool and language combination for which the library will fail to notice something is wrong. In the worst case it might hang, or confusingly claim a connection timeout occurred when really it just didn't understand the tool. Mitogen uses pickle on the network The crown jewel: it is absolutely true that Mitogen will pass untrusted pickles through an unpickler, with mitigations in place to prevent arbitrary code execution. But is it enough? Nobody knows for sure! It's impossible to combat this variety of FUD, but just in case a replacement serialization has existed in a local branch for over 6 months, waiting for just the day some material vulnerability is finally revealed. Be certain to mention pickle insecurity at every opportunity when generating negativity about the library, it's almost impossible to fight against it. The Ansible extension monkey-patches half the application Half may be a slight exaggeration, but indeed it's true, the Ansible extension is littered with monkey-patches. It doesn't matter that these are tidy and conservative patches that uninstall themselves when not in use or when incompatible situations are detected, monkey patching! is! wrong! and must! be prevented! Who could trust any code that subclasses types dynamically, or assigns a method to a class instance? No code should ever do that, or heaven forbid, nor should any test suite. Get this printed on a t-shirt so the message is abundantly clear. The Ansible extension reimplements the whole module executor It's true, it didn't even cutpaste the existing code, just blanket reimplements it! All spread out in a rats nest of 10 line methods across a giant inscrutible class hierarchy. The nerve! Despite replacing a giant chunk of Ansible's lower tiers, the plain truth is that this code is a weird derivative of existing Ansible functionality bolted on the side. Who in their right mind wants to run a setup like that? The Ansible extension prefers performance over safety Here's a great one: to avoid super important network round-trips when copying small files, rather than raise an error when the file copy fails, it instead raises the error on the follow-up module execution. How confusing! How can this mess possibly be debugged? The Ansible extension makes it easy for buggy modules to break things The dirty secret behind some of that performance? We're reusing state, the horror! And because we're reusing state, if some of that state is crapped over by a broken module, all hell could break loose. The sky might fall, hard disks could be wiped, fortune 500s might cease to exist! In the absense of collaboration with the authors of such evil code, Mitogen is forced between a rock and a hard place: maintaining a blacklist of known-broken modules and forking a threaded process (fork! with threads! the horror!) to continue the illusion of good performance in spite of absolutely no help from those who could provide it. Forking support is entirely disabled when targetting Python 2.4, and there is a high chance Mitogen will remove forking support entirely in the future, but in the meantime, use no less than 72 point bold font when discussing one of the library's greatest sins of all. The Ansible extension rewrites the user's playbook It's true! Why does a strategy module get to pick which connection plugins are used? It's as if the author believed in eliminating the need for unproductive busywork at all costs for the users of his project, and dared put user before code with a monkey-patch to ensure manual edits across a huge Git repo full of playbooks weren't necessary to allow the code to be tested, you know, by busy people who were already so stressed out about not having enough time to complete the jobs they've been assigned, due to a fundamentally slow and inefficient tool, that they'd risk experimenting with such a low quality project to begin with. The horror! Summary Mitogen has in excess of 4,000 interactive downloads (12,000 including CI) over the past year, of course not all of them users, but I have long since lost count of the number of people who swear by it and rely on it daily. In contrast to these numbers, GitHub currently sports a grand total of 23 open user-reported bugs. If you need more ammunition in your vague and futile battle with the future, please file a bug and I will be only too happy to supply it. But please, please, please avoid suggesting things "don't work" or are broken somehow without actually specifying how, especially in the context of a tool that continues to waste thousands of man hours every year. It is deeply insulting and a damning indictment on your ability to communicate and openly collaborate in what is otherwise allegedly the context of a free software project. Hopefully this post should arm such people with legitimate complaints for the foreseeable future. Meanwhile, I will work to improve the library, including as a result of bug reports filed in GitHub.

This an optimization story that should not surprise anyone using the Django ORM. But I thought I'd share because I have numbers now! The origin of this came from a real requirement. For a given parent model, I'd like to extract the value of the name column of all its child models, and the turn all these name strings into 1 MD5 checksum string. Variants The first attempted looked like this: artist = Artist.objects.get(name="Bad Religion") names = [] for song in Song.objects.filter(artist=artist): names.append(song.name) return hashlib.md5("".join(names).encode("utf-8")).hexdigest() The SQL used to generate this is as follows: SELECT "main_song"."id", "main_song"."artist_id", "main_song"."name", "main_song"."text", "main_song"."language", "main_song"."key_phrases", "main_song"."popularity", "main_song"."text_length", "main_song"."metadata", "main_song"."created", "main_song"."modified", "main_song"."has_lastfm_listeners", "main_song"."has_spotify_popularity" FROM "main_song" WHERE "main_song"."artist_id" = 22729; Clearly, I don't need anything but just the name column, version 2: artist = Artist.objects.get(name="Bad Religion") names = [] for song in Song.objects.filter(artist=artist).only("name"): names.append(song.name) return hashlib.md5("".join(names).encode("utf-8")).hexdigest() Now, the SQL used is: SELECT "main_song"."id", "main_song"."name" FROM "main_song" WHERE "main_song"."artist_id" = 22729; But still, since I don't really need instances of model class Song I can use the .values() method which gives back a list of dictionaries. This is version 3: names = [] for song in Song.objects.filter(artist=a).values("name"): names.append(song["name"]) return hashlib.md5("".join(names).encode("utf-8")).hexdigest() This time Django figures it doesn't even need the primary key value so it looks like this: SELECT "main_song"."name" FROM "main_song" WHERE "main_song"."artist_id" = 22729; Last but not least; there is an even faster one. values_list(). This time it doesn't even bother to map the column name to the value in a dictionary. And since I only need 1 column's value, I can set flat=True. Version 4 looks like this: names = [] for name in Song.objects.filter(artist=a).values_list("name", flat=True): names.append(name) return hashlib.md5("".join(names).encode("utf-8")).hexdigest() Same SQL gets used this time as in version 3. The benchmark Hopefully this little benchmark script speaks for itself: from songsearch.main.models import * import hashlib def f1(a): names = [] for song in Song.objects.filter(artist=a): names.append(song.name) return hashlib.md5("".join(names).encode("utf-8")).hexdigest() def f2(a): names = [] for song in Song.objects.filter(artist=a).only("name"): names.append(song.name) return hashlib.md5("".join(names).encode("utf-8")).hexdigest() def f3(a): names = [] for song in Song.objects.filter(artist=a).values("name"): names.append(song["name"]) return hashlib.md5("".join(names).encode("utf-8")).hexdigest() def f4(a): names = [] for name in Song.objects.filter(artist=a).values_list("name", flat=True): names.append(name) return hashlib.md5("".join(names).encode("utf-8")).hexdigest() artist = Artist.objects.get(name="Bad Religion") print(Song.objects.filter(artist=artist).count()) print(f1(artist) == f2(artist)) print(f2(artist) == f3(artist)) print(f3(artist) == f4(artist)) # Reporting import time import random import statistics functions = f1, f2, f3, f4 times = {f.__name__: [] for f in functions} for i in range(500): func = random.choice(functions) t0 = time.time() func(artist) t1 = time.time() times[func.__name__].append((t1 - t0) * 1000) for name in sorted(times): numbers = times[name] print("FUNCTION:", name, "Used", len(numbers), "times") print("\tBEST", min(numbers)) print("\tMEDIAN", statistics.median(numbers)) print("\tMEAN ", statistics.mean(numbers)) print("\tSTDEV ", statistics.stdev(numbers)) I ran this on my PostgreSQL 11.1 on my MacBook Pro with Django 2.1.7. So the database is on localhost. The results 276 True True True FUNCTION: f1 Used 135 times BEST 6.309986114501953 MEDIAN 7.531881332397461 MEAN 7.834429211086697 STDEV 2.03779968066591 FUNCTION: f2 Used 135 times BEST 3.039121627807617 MEDIAN 3.7298202514648438 MEAN 4.012803678159361 STDEV 1.8498943539073027 FUNCTION: f3 Used 110 times BEST 0.9920597076416016 MEDIAN 1.4405250549316406 MEAN 1.5053835782137783 STDEV 0.3523240470133114 FUNCTION: f4 Used 120 times BEST 0.9369850158691406 MEDIAN 1.3251304626464844 MEAN 1.4017681280771892 STDEV 0.3391019435930447 Discussion I guess the hashlib.md5("".join(names).encode("utf-8")).hexdigest() stuff is a bit "off-topic" but I checked and it's roughly 300 times faster than building up the names list. It's clearly better to ask less of Python and PostgreSQL to get a better total time. No surprise there. What was interesting was the proportion of these differences. Memorize that and you'll be better equipped if it's worth the hassle of not using the Django ORM in the most basic form. Also, do take note that this is only relevant in when dealing with many records. The slowest variant (f1) takes, on average, 7 milliseconds. Summarizing the difference with percentages compared to the fastest variant: f1 - 573% slower f2 - 225% slower f3 - 6% slower f4 - 0% slower

Across Visual Studio Code and Azure Notebooks, January brought numerous exciting updates to the AI and Machine Learning tooling for Python! This roll-up blog post recaps the latest products updates as well as the upcoming events for AI and Machine Learning: Python Interactive in VS Code now comes with IPython Console support Ability to work with Azure Machine Learning service in VS Code and Azure Notebooks Azure Notebooks Connect() News New resources Upcoming Events What’s new? Python Interactive in VS Code The Python extension for VS Code first introduced an interactive data science experience in the last Oct update. With this release, we brought the power of Jupyter Notebooks into VS Code. Many feature additions have been released since, including remote Jupyter support, ability to export Python code to Jupyter Notebooks, etc. The most noticeable enhancement in the Jan 2019 update allows code to be typed and executed directly in the Python Interactive window. Now, the window effectively turns into an IPython console that can be used standalone as well as in conjunction with the code editor. As well, the Jan 2019 update brings general Python goodness, such as improved diagnostics for failed tests with pytest and much faster outline view with the Python Language Server. See blog Python in Visual Studio Code – January 2019 Release for the full list. Work with Azure Machine Learning service in VS Code and Azure Notebooks To add some simplicity to the complex of world of ML pipelines, Microsoft announced the general availability of the Azure Machine Learning service in December. This service eases and accelerates the building, training, and deployment of your machine learning models from the cloud to the edge. The Azure Machine Learning extension for VS Code provides an easy way to manage the Azure Machine Learning service. This includes controls over experiments, pipelines, compute, models, and services, all from within VS Code. Interested? Check out how to get started with the AML extension. Additionally, Azure Notebooks offers a seamless way to take advantage of these new APIs within our hosted Jupyter notebook experience. You can join thousands of others who have tried out our Getting Started with the Azure Machine Learning service sample project . As well, you can connect directly with Azure Notebooks from your AML Workspace. Azure Notebooks Connect() News For those who missed our exciting Connect() Announcements, Azure Notebooks released new integrations with the Azure ecosystem as well as a fresh UI. With these improvements, we hope to help data scientists achieve greater productivity on our platform. For workloads that need a bit more power beyond on our free compute pool, Notebooks now allows you to connect to any SKU of Data Science Virtual Machine. Through this, users can take advantage of the full suite of Azure compute capabilities, right from Azure Notebooks. Read more in our documentation. We encourage users to try out these new compute options as well as the exciting Azure Machine Learning service integration mentioned above to further productivity on the Azure Notebooks platform. New resources Beyond feature releases, there are some great new demo resources from the Ignite Tour: Discovering Azure Tooling and Utilities covers many Azure tooling options including VS Code Running Machine Learning Experiments on Inventory Management covers a great machine learning use case utilizing Azure Notebooks and Machine Learning Services Upcoming Events Our Python tooling team as well as many of the Microsoft Cloud Developer Advocates have a few exciting events coming up: The Ignite Tour continues globally PyCascades is a new regional Python conference in the Pacific Northwest this weekend from February 23-24 Tell us what you think As always, we look forward to hearing your feedback! Please leave comments below or find us on Github (VS Code Python extension, Azure Notebooks)  and Twitter (@pythonvscode, @AzureNotebooks). The post What’s New with the AI and Machine Learning Tools for Python: February 2019 Update appeared first on Python.

In the last article, we started our discussion about the linked list. We saw what the linked list is along with its advantages and disadvantages. We also studied some of the most commonly used linked list method such as traversal, insertion, deletion, searching, and counting an element. Finally, we saw how to reverse a linked list. In this article, we will continue from where we left in the last article and will see how to sort a linked list using bubble and merge sort, and how to merge two sorted linked lists. Before we continue, it is imperative to mention that you should create Node and LinkedList classes that we created in the last article. Sorting a Linked List using Bubble Sort There are two ways to sort a linked list using bubble sort: Exchanging data between nodes Modifying the links between nodes In this section, we will see how both these approaches work. We will use the bubble sort algorithm to first sort the linked list by changing the data, and then we will see how we can use bubble sort to change the links in order to sort the linked list. Sorting Linked List by Exchanging Data To sort a linked list by exchanging data, we need to declare three variables p, q, and end. The variable p will be initialized with the start node, while end will be set to None. It is important to remember that to sort the list with n elements using bubble sort, you need n-1 iterations. To implement bubble sort, we need two while loops. The outer while loop executes until the value of variable end is equal to the self.start_node. The inner while loop executes until p becomes equal to the end variable. Inside the outer while loop, the value of p will be set to self.start_node which is the first node. Inside the inner while loop, the value of q will be set to p.link which is actually the node next to q. Then the values of p and q will be compared if p is greater than q the values of both the variables will be swapped and then p will point to p.ref, which is the next node. Finally, the end will be assigned the value of p. This process continues until the linked list is sorted. Let's understand this process with the help of an example. Suppose we have the following list: 8,7,1,6,9 Let's implement our algorithm to sort the list. We'll see what will happen during each iteration. The purpose of the bubble sort is that during each iteration, the largest value should be pushed to the end, hence at the end of all iterations, the list will automatically be sorted. Before the loop executes, the value of end is set to None. In the first iteration, p will be set to 8, and q will be set to 7. Since p is greater than q, the values will be swapped and p will become p.ref. At this point of time the linked list will look like this: 7,8,1,6,9 Since at this point of time, p is not equal to end, the loop will continue and now p will become 8 and q will become 1. Since again p is greater than q, the values will be swapped again and p will again become p.ref. The list will look like this: 7,1,8,6,9 Here again, p is not equal to end, the loop will continue and now p will become 8 and q will become 6. Since again p is greater than q, the values will be swapped again and p will again become p.ref. The list will look like this: 7,1,6,8,9 Again p is not equal to end, the loop will continue and now p will become 8 and q will become 9. Here since p is not greater than q, the values will not be swapped and p will become p.ref. At this point of time, the reference of p will point to None, and end also points to None. Hence the inner while loop will break and end will be set to p. In the next set of iterations, the loop will execute until 8, since 9 is already at the end. The process continues until the list is completely sorted. The Python code for sorting the linked list using bubble sort by exchanging the data is as follows: def bub_sort_datachange(self): end = None while end != self.start_node: p = self.start_node while p.ref != end: q = p.ref if p.item > q.item: p.item, q.item = q.item, p.item p = p.ref end = p Add the bub_sort_dataexchange() method to the LinkedList class that you created in the last article. Once you add the method to the linked list, create any set of nodes using the make_new_list() function and then use the bub_sort_dataexchange() to sort the list. You should see the sorted list when you execute the traverse_list() function. Sorting Linked List by Modifying Links Bubble sort can also be used to sort a linked list by modifying the links instead of changing data. The process remains quite similar to sorting the list by exchanging data, however, in this case, we have an additional variable r that will always correspond to the node previous than the p node. Let's take a simple example of how we will swap two nodes by modifying links. Suppose we have a linked list with the following items: 10,45,65,35,1 And we want to swap 65 and 35. At this point in time p corresponds to node 65, and q corresponds to node 35. The variable r will correspond to node 45 (previous to node p). Now if the node p is greater than node q, which is the case here, the p.ref will be set to q.ref and q.ref will be set to p. Similarly, r.ref will be set to q. This will swap nodes 65 and 35. The following method implements the bubble sorting for the linked list by modifying links: def bub_sort_linkchange(self): end = None while end != self.start_node: r = p = self.start_node while p.ref != end: q = p.ref if p.item > q.item: p.ref = q.ref q.ref = p if p != self.start_node: r.ref = q else: self.start_node = q p,q = q,p r = p p = p.ref end = p Add the bub_sort_linkchange() method to the LinkedList class that you created in the last article. Once you add the method to the linked list, create any set of nodes using the make_new_list() function and then use the bub_sort_linkchange() to sort the list. You should see the sorted list when you execute the traverse_list() function. Merging Sorted Linked List In this section we will see how we can merge two sorted linked lists in a manner that the resulting linked list is also sorted. There are two approaches to achieve this. We can create a new linked list that contains individually sorted lists or we can simply change links of the two linked list to join the two sorted linked list. In the second case, we do not have to create a new linked list. Let's first see how we can merge two linked lists by creating a new list. Merging Sorted Linked Lists by Creating a New List Let's first dry run the algorithm to see how we can merge two sorted linked list with the help of a new list. Suppose we have the following two sorted linked lists: list1: 10,45,65, list2: 5,15,35,68 These are the two lists we want to merge. The algorithm is straight forward. All we will need is three variables, p, q, and em, and an empty list newlist. At the beginning of the algorithm, p will point to the first element of the list1 whereas q will point to the first element of the list2. The variable em will be empty. At the start of the algorithm, we will have the following values: p = 10 q = 5 em = none newlist = none Next, we will compare the first element of the list1 with the first element of list2, in other words, we will compare the values of p and q and the smaller value will be stored in the variable em which will become the first node of the new list. The value of em will be added to the end of the newlist. After the first comparison we will have the following values: p = 10 q = 15 em = 5 newlist = 5 Since q was less than p, therefore, we store the value of q in em moved 'q' one index to the right. In the second pass, we will have the following values: p = 45 q = 15 em = 10 newlist = 5, 10 Here since p was smaller, we add the value of p to newlist, and set em to p and then moved p one index to the right. In the next iteration we have: p = 45 q = 35 em = 15 newlist = 5, 10, 15 Similarly, in the next iteration: p = 45 q = 68 em = 35 newlist = 5, 10, 15, 35 And in the next iteration, p will again be smaller than q, hence: p = 65 q = 68 em = 45 newlist = 5, 10, 15, 35, 45 Finally, p = None q = 68 em = 65 newlist = 5, 10, 15, 35, 45, 65 When one of the list becomes None, all the elements of the second list are added at the end of the new list. Therefore, the final list will be: p = None q = None em = 68 newlist = 5, 10, 15, 35, 45, 65, 68 The Python script for merging two sorted lists is as follows: def merge_helper(self, list2): merged_list = LinkedList() merged_list.start_node = self.merge_by_newlist(self.start_node, list2.start_node) return merged_list def merge_by_newlist(self, p, q): if p.item <= q.item: startNode = Node(p.item) p = p.ref else: startNode = Node(q.item) q = q.ref em = startNode while p is not None and q is not None: if p.item <= q.item: em.ref = Node(p.item) p = p.ref else: em.ref = Node(q.item) q = q.ref em = em.ref while p is not None: em.ref = Node(p.item) p = p.ref em = em.ref while q is not None: em.ref = Node(q.item) q = q.ref em = em.ref return startNode In the script above we have two methods: merge_helper() and merge_by_newlist(). The first method merge_helper() takes a linked list as a parameter and then passes the self class, which is a linked list itself and the linked list passed to it as a parameter, to the merge_by_newlist() method. The merge_by_newlist() method merges the two linked by creating a new linked list and returns the start node of the new linked list. Add these two methods to the LinkedList class. Create two new linked lists, sort them using the bub_sort_datachange() or the bub_sort_linkchange() methods that you created in the last section and then use the merge_by_newlist() to see if you can merge two sorted linked lists or not. Merging Sorted Linked Lists by Rearranging Links In this approach, a new linked list is not used to store the merger of two sorted linked lists. Rather, the links of the two linked lists are modified in such a way that two linked lists are merged in a sorted manner. Let's see a simple example of how we can do this. Suppose we have the same two lists list1 and list2: list1: 10,45,65, list2: 5,15,35,68 We want to merge them in a sorted manner by rearranging the links. To do so we need variables p, q and em. Initially, they will have the following values: p = 10 q = 5 em = none newlist = none Next, we will compare the first element of the list1 with the first element of list2, in other words, we will compare the values of p and q and the smaller value will be stored in the variable em which will become the first node of the new list. After the first comparison we will have the following values: p = 10 q = 15 start = 5 em = start After the first iteration, since q is less than p, the start node will point towards q and q will become q.ref. The em will be equal to start. The em will always refer to the newly inserted node in the merged list. p = 45 q = 15 em = 10 Here since p was smaller than the q, the variable em now points towards the original value of p and p becomes p.ref. p = 45 q = 35 em = 15 Here since q was smaller than p, em points towards q and q becomes q.ref. p = 45 q = 68 em = 35 Similarly em here points towards q. p = 65 q = 68 em = 45 newlist = 5, 10, 15, 35, 45 And here em points towards becomes p. p = None q = 68 em = 65 newlist = 5, 10, 15, 35, 45, 65 When one of the lists becomes None, the elements from the second list are simply added at the end. p = None q = None em = 68 newlist = 5, 10, 15, 35, 45, 65, 68 The script that contains functions for merging two lists without creating a new list is as follows: def merge_helper2(self, list2): merged_list = LinkedList() merged_list.start_node = self.merge_by_linkChange(self.start_node, list2.start_node) return merged_list def merge_by_linkChange(self, p, q): if p.item <= q.item: startNode = Node(p.item) p = p.ref else: startNode = Node(q.item) q = q.ref em = startNode while p is not None and q is not None: if p.item <= q.item: em.ref = Node(p.item) em = em.ref p = p.ref else: em.ref = Node(q.item) em = em.ref q = q.ref if p is None: em.ref = q else: em.ref = p return startNode In the script above we have two methods: merge_helper2() and merge_by_linkChange(). The first method merge_helper2() takes a linked list as a parameter and then passes the self class which is a linked list itself and the linked list passed to it as a parameter, to the merge_by_linkChange(), which merges the two linked by modifying the links and returns the start node of the merged list. Add these two methods to the LinkedList class. Create two new linked lists, sort them using the bub_sort_datachange() or the bub_sort_linkchange() methods that you created in the last section and then use the merge_by_newlist() to see if you can merge two sorted linked lists or not. Let's see this process in action. Create a new linked list using the following script: new_linked_list1 = LinkedList() new_linked_list1.make_new_list() The script will ask you for the number of nodes to enter. Enter as many nodes as you like and then add values for each node as shown below: How many nodes do you want to create: 4 Enter the value for the node:12 Enter the value for the node:45 Enter the value for the node:32 Enter the value for the node:61 Next, create another linked list repeating the above process: new_linked_list2 = LinkedList() new_linked_list2.make_new_list() Next, add a few dummy nodes with the help of the following script: How many nodes do you want to create: 4 Enter the value for the node:36 Enter the value for the node:41 Enter the value for the node:25 Enter the value for the node:9 The next step is to sort both the lists. Execute the following script: new_linked_list1. bub_sort_datachange() new_linked_list2. bub_sort_datachange() Finally, the following script merges the two linked lists: list3 = new_linked_list1.merge_helper2(new_linked_list2) To see if the lists have actually been merged, execute the following script: list3.traverse_list() The output looks like this: 9 12 25 32 36 41 45 61 Conclusion In this article, we continued from where we left in the previous article. We saw how we can sort merge lists by changing data and then my modifying links. Finally, we also studied different ways of merging two sorted linked lists.

Asus ROG Thor 1200W PSU Review: Asus Brings a Screen to the PSU Party  Tom's HardwareThe ROG Thor 1200W is Asus' new top PSU, made in collaboration with Seasonic. It features high performance and silent operation.

Market Reality: Samsung & Microsoft Dip into the Mobile AR Galaxy, with (Respectively) New Hardware & AR Apps  Next RealityThe augmented reality industry had enough twists this week to surprise even M. Night Shyamalan.

Boundary Scan Hardware Market 2019: By Product Type, Segmentation, Market Potential, Influential Trends, And The Challenges, And Forecast To 2024 – Daily Updates  Daily UpdatesThe Boundary Scan Hardware Market Report evaluate fundamental factors Along with the regional analysis, the current trends in the Boundary Scan Hardware ...

Microsoft workers demand it drop $480 million U.S. Army contract  ReutersSome Microsoft Corp employees on Friday demanded that the company cancel a $480 million hardware contract to supply the U.S. Army, with 94 workers ...

Samsung Shows Off Improved RF Hardware For 5G Networking  Android HeadlinesSamsung has now revealed that with its major smartphone announcements out of the way, it will be taking to Mobile World Congress 2019 in Barcelona to ...

PS5 Backwards Compatibility Patent Points To Emulation Power of Next PlayStation  NewsweekExecuting a “legacy application” on a “legacy device” may sound boring, but it may mean we're getting backwards compatibility for PS4 games on the next ...

IRS Find High FITARA Scores Stem from Good Software License Inventories  FedTech MagazineIt's one thing to get good grades, but when it comes to the FITARA scorecard, which publicly tracks how well federal agencies buy and manage technology, ...

Samsung's Multi-Experience store in Dubai Mall is an innovative first in experienced-based retail solutions

With the best gaming monitors, your games will come alive in ways you can’t even imagine.

We've dug deep to find only the best monitors in the US, UK and Australia.

All-in-one PCs come with Windows 10, macOS Sierra and ChromeOS these days. But which is best?

It might be time to get yourself one of the best computers of 2019, handpicked by our PC experts.

Equipped with the latest processors and graphics cards, these are the best gaming PC 2019 has brought forth.

Looking to upgrade your home cinema or start a new one? Check out these amazing projectors.